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ABSTRACT 

A 23-month evaluation of the Cooperative 
Demonstration Program (High Technology) assessed the extent to which 
grant applications selected for funding presented a clear and 
coherent design for the projects, project designs were successfully 
implemented, and project costs were reasonable in relation to 
projected or actual outcomes* Findings were based on data collected 
through a review of program records for 53 projects funded in FY 1988 
and FY 1989, mail or telephone surveys of key project staff of 39 
funded projects, and site visits to 27 funded projects. Of the 53 
funded applications analyzed, only 12 presented a clear and coherent 
project design, and an additional 26 were moderately clear and 
coherent in their design* Overall, projects focused on providing 
relatively short-term training. Projects had a high tech focus. The 
18-month time frame presented problems in completing project 
activities* Most projects did not conduct the kinds of evaluations 
that would allow possible replicators to determine whether the 
project was successful. The fact that the per-unit and per-outcome 
costs for 19 projects tended to cluster in the same area suggested 
that project costs were reasonably relative to one another. Ten 
lessons were suggested regarding project design, implementation, and 
costs* (Appendixes list nine references and describe the legislation 
establishing the program, the projects evaluated, and the results of 
the mail survey. (YLB) 
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PREFACE 



This document is the final report of a 23-nionth evaluation of the 
Cooperative Demonstration Program (High Technology). The Cooperative 
Demonstration Program (High Technology) was authorized under Title IV, 
Part B, Subpart 1 of the Carl Perkins Act of 1984 and was administered 
by the Office of Vocational and Adult Education (OVAE), U.S. Department 
of Education. The purpose of the evaluation was to determine the 
extent to which: (1) the grant applications selected for funding 
presented a clear and coherent design for the projects, (2) the project 
designs were successfully implemented; and (3) project costs were 
reasonable in relation to projected or actual outcomes. 

The report is based on data collected through a review of program 
records for 53 projects funded in FY 1988 and FY 1989, mail or 
telephone surveys of key project staff of 39 funded projects, and site 
visits to 27 of the funded projects. An essential ingredient to the 
success of these data collection efforts was the cooperation of the 
grantees' project staff. These people provided time to answer a 
lengthy mail survey and, in many cases, respond to questions from study 
team staff visiting the project. 

The study team consisted of staff members from COSMOS Corporation 
and Westat Corporation. Participating in the evaluation from COSMOS 
were: Peter Bateman (project director), June Sivilli, Lynne Adduci, 
and Judith Alamprese (corporate reviewer). Participating in the 
evaluation from Westat were: Lana Muraskin (deputy project director), 
Diane Steele, and Justin Boesel. The authors are grateful for the 
reviews and guidance from Gene Bottoms, Richard DiCola (OVAE), Robert 
Miller (OVAE), Bill Morrill, Roger Vaughan, and Tom White (OVAE), who 
served on the evaluation's advisory panel. Dr. Sandra Furey, Office of 
Policy and Planning, U.S. Department of Education, served as project 
officer for this study and provided valuable reviews and guidance. 

Among the authors, Peter Bateman prepared the Executive Summary 
and Sections I, II, V, and VI; Lana Muraskin and Justin Boesel prepared 
Section III and Appendix F; and Lynne Adduci prepared Section IV. 
Laura Baker and Roger Vaughan assisted in the editing of the report, 
and Priscilla Kates and Nique Murray typed the final manuscript. 

The opinions expressed in this report are those of the study team 
and do not necessarily reflect the opinions of the U.S. Department of 
Education. 
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EXECUTIVE SUMMARY 



The Cooperative Demonstration Program (High Technology) was 
established by Congress to support school districts, community col- 
leges, consortia, and private industry in demonstrating new approaches 
to vocational education. In FY 1988, the first year of the program, 
the Office of Vocational and Adult Education (OVAE), U.S. Department of 
Education, awarded 36 grants totaling $9.5 million. The following 
fiscal year, OVAE awarded an additional 30 grants totaling $9.2 
million. 



A> Purpose of the Cooperative Demonstration Program 

The Cooperative Demonstration Program (P.L.-524, Title IV, Part B, 
Subpart 1, Section 411) was the largest demonstration effort supported 
under the Carl D. Perkins Vocational Education Act of 1984 (the Act). 
The program provided the U.S. Department of Education (ED) and 
educational institutions an opportunity to try new approaches to 
vocational education and to learn about the effectiveness of these 
approaches. Funded projects were to reflect the Act's priorities: 

■ Increased access to high quality programs for 
special populations; and 

■ The overall improvement of the quality of 
vocational education. 

In addition to the authorized program activities specified in 
Section 411 of the Act, each year the Secretary of Education is 
authorized to designate additional program activities under the 
Cooperative Demonstration authority relating to the purpose of the Act. 
In the first year of the program (FY 1988), an invitational priority 
was issued for projects addressing high technology, but not all 
projects funded in FY 1988 responded to the invitation. In FY 1989, 
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high-technology was nade an absolute priority and a11 30 projects 
funded in FY 1989 addressed high-technology issues.' 

Projects also were: (1) to demonstrate successful cooperation among 
private employers and public agencies that resulted in training in 
advanced vocational education skills; (2) required to serve people 
enrolled in vocational programs directly; and (3) to be widely 
rep11cab1e by service providers. Furthermore, under Section 411 (b)(2) 
of the Act, grant recipients were required to provide, through cash or 
in-kind contributions, a minimum of 25 percent of the demonstration 
project's total cost. Contributions could include the fair market 
value of facilities, overhead, personnel, and equipment. 

B. Findings of the Evaluation 

This report presents the findings from an evaluation of high- 
technology projects funded by the Cooperative Demonstration Program in 
FY 1988 and FY 1989. The evaluation answered three important 
questions: 

■ Did the grant applications funded present a 
clear and coherent design for a project? 

■ Were the grantees able to implement their 
projects as proposed, and, if not, what 
problems affected their implemej^tation? 

/' 

■ Were project costs reasonable in relation to 
projected (those contained in the 
application) or actual outcomes? 

Clear and Coherent Proiect Design . Of the 23 FY 1988 funded 
applications analyzed, only three presented a clear and coherent 



*In an absolute priority, a focus on high technology is required, while in 
an invitational priority, a focus on high technology is encouraged but not 
required. 
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project design while an additional ten were moderately dear and 
coherent in their design, as a result of the Eva1uab111ty Assessment 
conducted by the study team. These applications were judged to be, 
respectively, h1gh1> 11ke1y and moderately 11ke1y to be successful. 
The remaining ten applications were much less dear In their design 
and, accordingly, judged much less likely to be successful. 

The quality of the awarded applications improved In the second 
year of the program^ Of the 30 FY 1989 applications reviewed, nine 
presented a clear and coherent design, while an additional 16 were 
moderately clear and coherent In their design as a result of the 
Evaluablllty Assessment conducted by the study team. The remaining 
five applications were much less clear In their design and judged less 
likely to be successful. 

Project Implementation . The evaluation focused on five 'aspects of 
project Implementation: project training, public/private partnerships, 
the Involvement with high technology, the operation of the project 
within the grantee Institution, and the repllcablllty or 
••exportablllty'^ of the projects' activities and products. 

Project Training . Overall, projects focused on providing rela- 
tively short-term training. The typical participant received fewer 
than 100 hours of training. The projects included In the site visits 
can be grouped Into a few basic categories: 



I Short-term, ski 11 -specific Instruction for 
current employees of companies* or for persons 
already familiar with the field of training 
(this approach was most common among 
manufacturing projects but could also be seen 
in the refresher nursing course and in 
business projects teaching specific software 
applications, as well as home automation 
installation); 

■ Medium-term, entrv-level technician training 
ranged from a few months to a year, often 
aimed at helping unemployed or otherwise 
disadvantaged persons find jobs in a 
particular field. (These programs cut across 
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the Industry lines but were quite coninon in 
business); and 

■ Lonacr-tftrm programs for two-vear certi- 
fication or degrees were aimed at 
institutional capacity-building and regular 
students, e.g.^ creating new or revising 
existing offerings within departments of 
industrial technology or business/computers 
In coMMinity colleges. (One was an appren- 
ticeship training project, and one an occu- 
pational program for high school students.) 



Partnerships . Intensities of partnerships varied from two 
projects where single employers played critical roles in service 
development and delivery to several projects in which employers did 
little more than attend an advisory committee meeting. According to 
survey results, most partnerships fit one of two models: 



■ The partner was a customer of the project, 
e.g. the project provided customized training 
to the partner's employees; or 

■ The partner shared actively in the delivery 
of instruction and services. 



Projects with multiple partners were considerably more likely to 
report that partners were customers, while projects with few partners 
used partners in more ways, including active sharing of instruction. 

High Technology . The U.S. Department of Education's initial 
concerns that projects might not have a high-tech focus do not appear 
to have been reflected in practice. This is true primarily because 
almost all fields have some high-tech elements. 

Operation and Integration of Pro.i ects within Institutions. FY 
1989 projects reported fewer start-up problems than did FY 1988 
projects. The 18-month time frame, however, continued to present 
problems in completing project activities. Whether a project started 
providing services shortly after an award made little difference in 
whether it continued after the grant ended. The site visit teams 
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identified several problems that cut across projects* These included 
an inability to recruit special populations as planned, problems in 
getting and maintaining facilities and materials, poorly developed 
designs that could not be executed, and underestimating the time 
necessary for product development. Aside from the recruitment 
difficulties, most problems were resolved but had an impact on those 
projects* ability to complete work within the 18-month grant time 
frame. 

ExDortabilitv of Project Activities and Products . Projects lacked 
consensus on what it Mas that the Cooperative Demonstration Program was 
"demonstrating." Only 20 projects considered their public- 
partnerships, and only 13 projects considered their training as 
applicable to, or providing a useful model for, others. Most projects 
simply did not conduct the kinds of evaluations that would allow 
possible replicators to determine whether the project was successful 
for participants. Projects that developed and tested discrete 
curricula or products were more likely to have some evidence of 
effectiveness and a product to disseminate. 

Project Costs . The evaluation team analyzed the three activities 
for which cost data were available through the grantee's accounting 
system or the final contract budget: planning and administration, 
student training, and curriculum development. 

Treatment Costs . The first major analysis separated planning 
costs from the costs of providing the service. The proportion of total 
project resources devoted to planning and administration ranged from 10 
percent to 48 percent. 

Project Intensity . The intensity of the project is the number of 
successful outcomes relative to the effort expended to accomplish those 
outcomes. Project intensity for each activity for which data were 
available-training students and training staff-ranged from 17 hours to 
7,000 hours. Staff training, offered formally at only three sitec, 
ranged from 20 hours to 54 hours per teacher. 

Average Unit Cost of Services . The average cost per unit of 
service is the total cost of the service divided by the number of units 
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provided. Total units of service provided was the sum of a11 students 
entering training or the number of new courses. The average unit cost 
for student training (cost per training hour) ranged from $.45 to 
$40.47. 

Comparing the average cost-per-hour-of-training across projects 
may create an unfair comparison because of variations in the intensity 
of the training and the number of students being trained. These dif- 
ferences affect the comparison of costs for curriculum development. 
The average unit cost for curriculum development (cost per course hour) 
ranged from $2.00 to $24,180. However, comparing the average cost-per- 
unit-hour for curriculum development also may be misleading. One 
project's course was an interactive videodisc, which had a much higher 
initial development cost due to the technology used. 

Service Cost Per Unit of Outcome . The unit service costs for 
training students (the cost per student) ranged from $190 to $62,971. 
For curriculum development, the cost per course ranged from $233 to 
$241,799. The relatively high cost at one project again was due to the 
high cost of the interactive videodisc. 

In summary, the answer to the question "are project costs reason- 
able in relation to project outcomes?" appears to be yes for all pro- 
jects. Neither OVAE nor the literature of vocational education have 
defined the absolute cost standards against which to measure ^'reason- 
ableness" of demonstration activities such as training or curriculum 
development. Thus, it is not possible to conclude that project costs 
were either reasonable or unreasonable in an absolute sense. However, 
the fact that the per-unit and per-outcome costs for 19 of the projects 
tended to cluster in the same area (even though total costs and project 
intensity varied substantially) suggests that project costs were 
reasonably relative to one another. 
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C. Lessons for Federal Program and Demonstration Management 

Although the Cooperative Demonstration Program (High Technology) 
has ended, the evaluation of the program has yielded important lessons 
that can be applied to future demonstrations sponsored by OVAE and the 
Department. The information obtained from the 27 FY 1988 and FY 1989 
grantees visited by the study team and the 30 FY 1989 grantees surveyed 
suggest ten lessons regarding project design, project implementation, 
and project costs. 



Lesson 1 : 

Program regulations should be more narrowly defined to convey 
explicit information about the kinds of interventions sought. 



Although titled "Cooperative Demonstration Programs," the autho- 
rizing legislation did not clearly define the nature of the demon- 
stration. It was not clear whether the funded projects were to demon- 
strate that a particular program can be successful if it has not been 
tried before, can be improved in its original site if already opera- 
ting, can be successful in a new site if already implemented elsewhere, 
or some combination of these intentions. The broad definition of 
acceptable interventions allowed a wide variety of projects to be 
funded. Although all projects satisfied the general conditions of the 
program's regulations, it is uncertain whether all projects met the 
intent of the regulations and the Perkins Act. 



Lesson 2: 

Applicants should be required to show the logic and 
plausibility of their project designs. 



In general, demonstration projects are likely to be most success- 
ful if they propose a logical design of activities to be accomplished 
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and objectives to be achieved. A logical prcject design identified the 
linkages between resources and activities, activities and short-tr^rm 
outcomes, and short-term outcomes and long-term outcomes. Applicants 
for Cooperative Demonstration grants often were unclear in their logic 
regarding how resources would be used to conduct the activities. 
Future applicants should be required to explicitly show the logic cf 
the project design. 

A pi ausible project design has some likelihood of achieving its 
short-term objectives, where short-term is defined as the grant award 
period. Project objectives may be implausible because: (1) schedules 
are unrealistic; (2) resources are insufficient; or (3) available know- 
ledge suggests that the project is not likely to achieve its objectives 
(e.g., objectives may be too ambitious or the staff may not possess the 
requisite skills). As with logic, applicants should be required to be 
explicit regarding the plausibility of the project design. 

The panel reviewing the applications should include experts who 
are familiar with the high technology field or activities being pro- 
posed and who can judge plausibility. Such experts could identify 
activities (e.g., the development of interactive videodiscs), which are 
not likely to be feasible within the time frame or resources proposed. 
The experts also could assess whether the applicant's experience in a 
field is sufficient to accomplish successfully more complicated activi- 
ties, e.g., the construction of a Class I clean room for manufacturing 
computer chips. 

Furthermore, if logic and plausibility are important to the 
success of the project, OVAE should award points for them in the 
evaluation criteria. Applicants take their cue from the point award 
criteria in the grant application package to emphasize certain areas 
over others. No points were given for logic and plausibility, even 
though these have a serious impact on potential success. 
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Lesson 3: .. . 

: ;-:Sraht^^ project starts in 

the^Ciimher greater availability i)^^^^^^ staff, 

iredii^^^^ tf ai ning deinahds by other programsy • and w opportun Hies 
for planning*- V ■ ''y::^w 



The timing of the award notification and starting date has impor- 
tant implications for project implementation. Most academic institu- 
tions plan their staffing and activities according to the academic-year 
calendar. Academic institutions plan in the spring for the following 
year and try to have both activity schedule and staff assignment's 
SPttled by May or June preceding the September start. Grants that 
start at the beginning of the academic year seem to be easier to 
implement than those starting midyear. 

The major exception to scheduling project start-up for the begin- 
ning of the academic year is curriculum development projects that use 
significant amounts of instructor time to develop courses. 



Lesson 4 : 

the applicant's proposed training should be appropriate to the 
labor heeds of the ^^^^^^^ surrounding the institution based 

on mdit recent Tabor market data. 



The labor market data used by most applicants described potential 
job openings and economic conditions for the state or region, rather 
than for the county or city in which the institution was located. 
These labor market data often were one to three years old at the time 
of the application. Although the data may have demonstrated a need for 
job training, the jobs may have been located farther away than students 
were willing to commute (or relocate). 
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: ■ Appllcan^^^^^ in detail the role and respbrisibillties 

of the private parth^^^^ Include a statement from the partner 

agreeing to accept thi s rol e . 




The specific role of the private sector was not well defined in 
the program regulations and the private sector role varied widely 
across projects. Projects with multiple partners were considerably 
more likely to report that partners were customers* while projects with 
few partners used partners in more ways, including active sharing of 
instruction. 

The vagueness in the applications of how private companies would 
be involved in their projects sometimes reflected incomplete planning 
on the part of the grantee. Other times it was the result of the 
grantee not being able to get the private partners to commit to 
training students or to activities other than advising the project. 
The applicant should be required to submit a letter from each partner 
stating that it is willing to commit to the level and type of support 
stated in the application. After the award, and during the planning 
stage, the grantee should establish a formal agreement with the partner 
detailing precise responsibilities. 



Lesson 6 ; 



Grantees should be encouraged to request changes to the scope 
of work in response to changes in the local labor market or other 
conditions* 



Several projects encountered operational problems during various 
stages of the grant, including a deteriorating or shifting labor 
market, withdrawal of a key partner, or discovery that an activity was 
more complicated or expensive than originally estimated. In reviewing ' 
the implementation history of the projects, the study team thought some 
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problems were severe enough to warrant a change in project design or 
objectives, although few such changes were requested. 

During the site visits, the teams asked project directors and 
staff why they did not request changes in grant scope to keep up with 
changing conditions. Almost all the project directors said that they 
thought they had to fulfill the terms of the application and that any 
requests for changes would be considered a sign of failure and/or not 
approved. This perception on the part of grantees resulted in projects 
continuing on an inappropriate— and sometimes unsuccessful— path. 



Lesson 7 : 

The length of the grant should allow sufficient time for 
project start-up and accomplishment of the stated objectives. 



The duration of the grant has implications both for the activities 
that can be carried out and the ability to draw conclusions based on 
grant outcomes. The time period for the Cooperative Demonstration 
grants was 18 months, with some projects requesting three- or six-month 
extensions. For grantees who began new projects or had no existing 
staff to assign to the project, between three and six months of the 
grant period was devoted to hiring staff, establishing office and 
contracting procedures, and planning training. For grantees with 
existing staff or a similar training activities program already in 
place, start-up time was not as great, and the training offered was 
often more extensive (e.g., part of an ongoing two-year training 
program). Almost all grantees agreed that 18 months was not enough 
time to complete training other than customized training. 



Lesson 8: 

urantees should limit the number of objectives and be held more 
accountable for meeting the objectives stated in their applicatiohs 
or revised scopes of work. 
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Applicants tended to state numerous objectives phrased In ways 
that made It difficult to measure success. Often the objectives 
referred to conditions that required more time than the grant period 
(e.g., produced journeyman technicians), required changes beyond the 
scope of the grant (e.g., make local employers more competitive), or 
were not time specific (e.g., meet employer's demand for trained 
workers). Objectives should be stated In operational terms that can be 
measured by third-party evaluators, and the evaluations should focus on 
how well projects met those objectives. 



lesson 9 ; 



Applicants should better substantiate the value assigned to In- 
kind contributions. 



The non-Federal contribution by grantees consisted primarily of: 
(1) grantee staff time and classroom space; and (2) partner staff time 
and equipment donations. While the grantee staff time and space could 
be easily documented, the value assigned to partner staff and equipment 
contributions was simply self-declared without an Independent 
appraisal . 

In reporting the self-declared value of the donated equipment, 
grantees failed to Include In their proposed budget the cost of trans- 
portation. Installation, maintenance, and supplies for the equipment. 
These additional, unplanned costs sometimes required the reallocation 
of grant funds after the project started or prevented the grantee from 
utilizing the equipment as proposed. Encouraging applicants to recog- 
nize all the costs associated with accepting donated equipment should 
help prevent a shortage of resources during implementation. 
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Grantees should be encouraged to build upon the existing 
llter^lture, curricula, and other educational resources available Iri 



Many of the projects engaged in curricula development either as a 
primary objective of the grant or in response to the requests for 
specific training from a partner organization. Rather than searching 
for existing course materials through ERIC or the OVAE curriculum 
centers and adapting these materials to the needs of the project, many 
projects developed their own curricula from scratch. Project directors 
reported they did not use— or at least review— the curriculum materials 
available elsewhere because: (1) they considered their training to be 
"unique** and that materials developed elsewhere would not be relevant; 
or (2) they were unaware that applicable materials were available else- 
where. A few project directors reported they checked with other 
grantees, but none reported conducting a systematic search of ERIC, the 
curriculum centers, or other clearinghouses. 
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K INTRODUCTION 

This report summarizes the activities of, and findings from, an 
evaluation of projects funded by the Cooperative Demonstration Program 
(High Technology) in FY 1988 and FY 1989* The evaluation answers three 
questions: Did funded grant applications present a clear and coherent 
design for a project? Were grantees able to implement their projects 
as proposed, and, if not, what problems prevented implementation? Were 
project costs reasonable in relation to projected or actual outcomes? 

Overview of the Cooperative Demonstration Program 

The Cooperative Demonstration Program (P,L, 98-524, Title IV, Part 
B, Subpart 1, Section 411) was the largest demonstration effort 
supported under the Carl D. Perkins Vocational Education Act of 1984 
(see Appendix A for a copy of the legislation). The program was 
reauthorized in 1990 (P.L. 101-392) and continues to provide the U.S, 
Department of Education (ED) and educational institutions, through the 
grantees, an opportunity to try new approaches to vocational education 
and to learn about the effectiveness of these approaches. In 1988 and 
1989, funded projects reflected the Act*s priorities: increased access 
to high quality programs for special populations and overall 
improvement of the quality of vocational education. Projects also were 
to demonstrate successful cooperation among private employers and 
public agencies that resulted in training in advanced vocational 
educational skills. The regulations at 34 CFR Part 412 suggested a 
variety of models, including: work experience and apprenticeship, 
worksite training, placement, and public works. Agencies eligible to 
apply included State Education Agencies (SEAs), Local Education 
Agencies (LEAs), postsecondary educational institutions, institutions 
of higher education, and other public and private agencies, 
organizations, and institutions. 

In 1988 and 1989 (as continues to be the case today), projects 
were authorized to be funded through grants, cooperative agreements, or 
contracts, and could be: 
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Node! projects providing improved access to 
quality vocational education programs for- 

-*handicipped individuals; 
-disadvantaged individuals; 
-adults who are in need of training 
and retraining; 

-individuals who are single parents 
or homemakers; 

-Individuals who participate in programs 
designed to eliminate sex bias and 
stereotyping in vocational education; 

-crininal offenders who are serving 
in a correctional institution; and 

-men and women seeking nontraditional 
occupations* 



Projects that were examples of successful 
cooperation between the private sector 
(including employers, consortia of 
employers, labor organizations, and 
building trade councils) and public 
agencies in vocational education, including 
State boards and eligible recipients* The 
projects had to be designed to demonstrate 
ways in which vocational education and the 
private sector of the economy could work 
together effectively to assist vocational 
education students to attain the advanced 
level of skills needed to make the 
transition from school to productive 
employment, including- 

-work experience and apprenticeship 
projects; transitional worksite job 
training for vocational education students 
that is related to their occupational 
goals and closely linked to classroom and 
laboratory instruction provided by an 
eligible recipient; 

-placement services in occupations that the 
students are preparing to enter; and 

-where practical, projects that will 
benefit the public, such as the rehabili- 
tation of public schools or housing in 
inner cities or economically depressed 
rural areas. 
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The projects are authorized to include 
institutional and on-the-job training, 
support services authorized by the Act, 
and such other necessary assistance as the 
Secretary determines to be necessary for 
the successful completion of the project. 



3. Projects to overcome national skill 

shortages, as designated by the Secretary 
in cooperation with the Secretary of Labor, 
Secretary of Defense, and Secretary of 
Commerce. 



4. Such other activities that the Secretary 
could designate, which are related to the 
purposes of the Act. r Federal Register . 
1985, pp. 33260-33261.] 

All projects, however, had to serve directly people enrolled in 
vocational programs and be widely replicable by service providers. 
Furthermore, grant recipients had to provide, through cash or in-kind 
contributions, a minimum of 25 percent of the demonstration project's 
total cost. Contributions could include the fair market value of 
facilities, overhead, personnel, and equipment. 

In addition to the specified activities contained in the Act, 
under the Education Department General Administrative Regulations, the 
Secretary of Education has authorized each year to establish priorities 
for the program. In the first year of the program (FY 1988), an 
invitational priority (in addition to the absolute priority for 
authorized activities) was issued for projects addressing high 
technology. In FY 1989, high technology became an absolute priority. 
The term "high technology" Was defined by the Act to mean: 

...state-of-the-art computer, microelectronic, 
hydraulic, pneumatic, laser, nuclear, chemical, 
telecommunication, and other technologies being 
used to enhance productivity in manufacturing, 
communication, transportation, agriculture, mining, 
energy, commercial, and similar economic activity. 
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and to improve the provision of health care, 
(Section 521(16) of the Act; 34 CFR 400-4(b)), 

The Application Process 

Projects funded by the program were proposed and implemented by 
educational institutions, private ;»Qencies, and other organizations. 
Each year since the program's inception in FY 1988, the Office of 
Vocational and Adult Education (OVAE), U.S. Department of Education, 
received grant applications from agencies, institutions, and 
organizations interested in conducting demonstrations. Program 
application requirements, the high technology, invitational priority, 
and selection criteria were published in the Federal Register , In FY 
1988, application materials were mailed directly to prospective 
applicants; in FY 1989, OVAE staff instructed applicants to photocopy 
the application and instructions contained in the Federal Register , 

In general, the application process was as follows. Applicants 
prepared and submitted project applications according to the published 
requirements priority and selection criteria. Panels of outside 
readers, selected by OVAE, reviewed the applications and asked 
applicants for any necessary clarification. The applications were 
judged according to the following selection criteria and point 
allocations: 

■ Statement of noed (15 points); 

■ Plan of operation (30 points); 

■ Quality of key personnel (10 points); 

■ Budget and cost effectiveness (10 points); 

■ Evaluation plan (5 points); 

m Adequacy of resources (5 points); 

■ Private-sector involvement (10 points); 

■ Employment opportunities (5 points); and 

■ Dissemination (10 points). 

The projects receiving the highest scores were awarded grants, and as 
many projects were funded as the program budget would allow. In FY 
1988, a total of $9,5 million was awarded, and in FY 1989 grant awards 
totaled $9,2 million. 
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Of 181 applications submitted in the programs's first year (FY 
1988), 36 were approved and given grants ranging in size from $50,000 
to $550,000. The winning projects were notified in October 1988 and 
most began their work around January 1989. Although operating in 
calendar years 1989 and 1990, the first cohort of projects are 
considered FY 1988 projects because funding was provided from the FY 
1988 Perkins Act appropriation. 

In FY 1989, 106 applications were submitted and 30 received 
grants. Most of these projects began in January 1990, although at 
least two grantees who had received grants under the FY 1988 
competition delayed starting until July 1990 in order to complete their 
FY 1988 projects. 

Not all FY 1988 projects had responded to the Secretary's 
invitational priority of addressing high-technology issues. OVAE 
identified only 23 of the 36 funded projects as being "high 
technology," based either on the type of job for which training was 
conducted (or curriculum developed) or on the nature of the training 
given students. These 23 projects were the focus of the first-year 
evaluation effort. All 30 projects funded in FY 1989 addressed high- 
technology issues and were the focus of the second year evaluation 
effort. The lists of the FY 1988 and FY 1989 high technology projects 
are presented in Appendix B-1 and Appendix B-2. 

Defining Project Boundaries 

For this evaluation, "project" was defined as the activities 
funded by the Cooperative Demonstration grant and non-Federal matching 
funds and occurring within the 18-month grant period. It did not 
include services provided before or after the grant period and benefits 
accruing after the end of the grant. Also excluded were the training 
activities and support services offered to project enrol lees through 
other parts of the grantee institution. For example, a participant 
might enroll in training developed with project funds and provided by a 
project-supported instructor, using equipment donated by a local 
business as part of the non-Federal "match." Yet, to complete the 
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degree or certificate for which the training was developed, the student 
might be expected (by the project design) to enroll in additional 
courses in the same institution that were not supported by the project. 
Students might also use financial aid or support services provided by 
or through the institution that were not part of the project. While 
these activities might be necessary to complete the degree or 
certificate, only those directly funded by the grant were included in 
this evaluation. 

Boundaries of the project include interorganizational networks 
developed in the course of carrying out a project. Grantee 
institutions entered into a variety of formal and informal relation- 
ships with other organizations to offer students support services, 
provide them with jobs, or provide the project with additional 
financial or other assistance. 

Overview of this Reoort 

This final report is presented in six sections. Section I 
introduces the Cooperative Demonstration Program and the three major 
questions in the evaluation. Section II describes the modified 
evaluability assessment done for the 30 FY 1989 projects and answers 
the first study question: were projects clearly and coherently 
designed? Section III describes the implementation of a subset of FY 
1989 projects, based on a mail survey of 30 projects (27 projects 
responded) and site visits to 19 projects. Section IV analyzes project 
costs relative to project accomplishments and answers the third study 
question: were project costs "reasonable?" Section V examines two 
groups of issues related to the management of the Federal program 
raised by the evaluation and suggests ways to improve future 
demonstration programs. Finally section VI summarizes the findings of 
the first four sections and recommends possible improvements in 
programs. The nine appendices describe the projects evaluated, the 
results of the mail survey, and the survey and field instruments. 
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Section II 

AN ASSESSMENT OF PROJECT LOGIC AND DESIGN 



II-l 

II. AN ASSESSMENT OF PROJECT LOGIC AND DESIGN 



This section ranks the clarity and coherence of 30 grant applica- 
tions funded under the Cooperative Demonstration Program (High Tech- 
nology) In FY 1989. Each grantee application was assigned a score 
according to its internal logic and the plausibility of Its design. 
Originally, this evaluation had intended to relate the applications' 
scores to the ultimate success of the projects. Of the 30 grant appli- 
cations reviewed for logic and plausibility, four projects were 
predicted to be highly successful, 21 projects were predicted to be 
potentially successful, and five projects were predicted to be less 
successful. Ideally, only applications that offered well-defined sets 
of services and the possibility of success should be funded. 

Predicting the likelihood for success of the proposed projects can 
' be accomplished, in part, using a methodology known as an evaluability 
assessment (EA). While EAs are traditionally used to determine the 
most appropriate design for a program evaluation, parts of the metho- 
dology can be useful for determining the logic or clarity and the 
plausibility or coherence of individual project designs. Part A 
describes the procedures used to develop a logic model for each funded 
project and to characterize the logic of the proposed design. Part B 
describes the procedures used to score the plausibility of each project 
design. Part C ranks the projects on both logic and plausibility to 
arrive at an overall prediction of success for the project. 
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A. Assessing the Logic of the Project Design 



The first part of the assessment created a logic model for each 
proposed project. Logical designs include well-defined project 
objectives and activities that are linked to those objectives. 
Projects are nore likely to succeed if they propose a dear and logical 
design of objectives to be achieved and activities to be accomplished. 
Designs can be detailed but illogical if activities do not proceed 
systematically from objectives. Unclear designs, however, cannot be 
logical because there is insufficient information to trace the logic 
linking the components. Projects may be uneven, with some dear and 
logical components and others that are not. 

To determine each application's logic, the study team reviewed 
each grant application to identify the following: 

■ an explicitly stated set of inputs or 
resources; 

■ an explicitly stated set of activities or 
events; 

■ an explicitly stated set of outcomes (short- 
term and long-term); and 

■ an explicitly stated set of causal links 
among events and outcomes that establish the 
flow of effects expected from the project. 

The study team recorded the components of each of the 30 grant 
applications in a logic shell illustrating the linkages. Two 
contrasting examples of logic models illustrating the procedure are 
provided. Figure II-l displays a logic model where several events are 
linked with project outcomes, demonstrating a clear and logical design. 
In contrast. Figure I 1-2 displays a project design where no such 
linkages are evident. Scoring each design reflects whether the project 
posses a logical design according to procedures described below. 
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Developing the Logic Model 

The study team followed a six-step process when reviewing grant 
applications provided by the OVAE program office. The team assembled 
all material pertaining to the grant application and proposal, 
including: the original proposal; any revisions to the original 
proposal; correspondence dated prior to the award date of the grant; 
supplementary material submitted by the applicant prior to the date of 
award; and grant award documents. 

Next, the team identified long-term outcomes from information 
contained in each application's introduction, statement of need, and 
objectives. Long-term outcomes were entered in the logic model shell. 
The team recorded all information as stated in the application and, 
when information was available, specified who the project was intended 
to serve. The team next entered short-term objectives from information 
in the "plan of operation" section of the application. Then the team 
entered the resources available to the project on the first day of the 
grant award from the budget, introduction, and other parts of the 
application. 

Finally, the team showed the stated causal links between inputs, 
activities, short-term goals, and long-term goals with directional 
arrows, based on reviewing budget narrative for explanations of how 
Federal and non-Federal funds were to be spent. A directional arrow 
was drawn when the narrative matched activities specified in the 
diagram. The team reviewed the remainder of the application for other 
logical relationships stated explicitly within the same paragraph in 
the proposal. Because paragraphs should represent complete thoughts, 
linkages were designated only if two elements were expressed within the 
same paragraph. This is a restrictive approach and affects the 
projects' final rankings. Without such restrictions, however, the team 
could not determine objectively if or how the applicants intended to 
connect the activities. Boxes without connecting arrows represent 
logical gaps. 
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Scoring Pr oject Logic 

Procedure , Applications were scored in a three-step process • 
First, the study team counted the number of inputs, activities, short- 
term outcomes, and long-term outcomes. Totals for each are shown in 
the summary of "Raw Scores of Project Logic," presented in Appendix 
C-1. 

Then the team determined what activities were identified for 
achieving each short-term outcome. The team assigned one point for 
each short-term outcome supported by at least one activity and 
subtracted one point for each short-term outcome not so supported. 

Next, the team determined the short-term outcomes identified for 
achieving each long-term outcome. Again, the team assigned one point 
for each long-term outcome supported by at least one short-term outcome 
and subtracted one point for each long-term outcome not so supported. 

Analysis . The design of the project was considered clear and 
logical if it met five conditions. First, the design must have 
identified at least one long-term outcome, e.g., access to quality 
vocational education training or enhancement of the competitive 
strength of local industry. Second, the design had to identify one or 
more short-term outcomes to be achieved during the grant period that 
would help achieve long-term outcomes. Third, the design had to 
specify one or more activities during the grant period that would help 
achieve a short-term or long-term outcome. Fourth, the design had to 
have had more linkages than gaps between activities and short-term 
outcomes. Total points assigned to linkages or gaps, therefore, must 
yield a positive number to be considered logical, e.g., a project that 
had four explicit linkages and two gaps had a total score on this 
criterion of +2. Fifth, the design had to have more linkages than gaps 
between short-term outcomes and long-term outcomes. The total points 
assigned to linkages or gaps had to be a positive number, e.g., a 
project that had four stated linkages and three gaps had a total score 
on this criterion of +1. Raw scores for the projects are shown in 
Appendix C-1. 
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Results of the Logic Assessment 

Descrlotion of Project LoqIc , Using these procedures, the study 
team assigned each project an overall score for logic. After testing 
for the presence of all five conditions, it found: 

■ All 30 projects identified one or more long- 
term objectives; 

■ Eleven projects linked one or more short-term 
outcomes to a long-term outcome; 

■ Twenty-nine projects linked one or more 
activities to a short-term outcome; 

■ Twenty-three projects displayed more linkages 
than gaps between activities and short-term 
outcomes; and 

■ Only two projects displayed more linkages 
than gaps between short-term and long-term 
outcomes. 

Table II-l ranks each project according to the five conditions and 
shows total scores. Based on the overall scores, nine projects 
received a high rating; 16 received a medium rating; and five received 
a low rating. Table II-2 summarizes the logic rankings of all the 
projects. 

Most applications met two criteria: they identified long-term 
outcomes and linked proposed activities to short-term outcomes. To 
demonstrate a logical design, however, the proposals also needed to 
identify linkages between short-term and long-term outcomes. Only two 
projects had more linkages than gaps between short-term and long-term 
outcomes. Further, application's often were unclear about how 
resources would be used to conduct the activities. Many projects 
indicated that Federal funds would be used to hire staff. They often 
failed, however, to report precisely what staff would be doing. In 
summary, most applications did not explicitly link resources with 
activities. 
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B, Assessing the Plausibility of Project Design 



Plausibility Defined 

A plausible project is one with some likelihood of achieving its 
short-term objectives (short-term is defined as the 18 months of the 
grant award period)* Project objectives may be implausible because 
schedules are unrealistic, resources are insufficient, or available 
knowledge suggests that the projects are unlikely to achieve objectives 
(e.g., too ambitious, or unskilled staff). Thus, assessing the 
plausibility of the project's design means determining the degree to 
which a project: 

■ Is we11 defined; 

■ Describes feasible relationships among 
components; 

■ Can be completed with available resources; 
and 

■ Can be completed within the specified time 
period. 

For each major short-term outcome, the study team ?,sked the 
following questions: Are resources adequate to achieve outcomes? Is 
the schedule for achieving the outcome reasonable? Do the activities 
suggest an understanding of the steps necessary to achieve the outcome? 
The answers to these questions were recorded in Table I 1-3. 

Procedures for Assessing Plausibility 

The team followed a five-step process to determine project 
plausibility. First, it reviewed the logic models already developed as 
well as the grant application to determine whether resources were 
adequate for each activity. One point was assigned to each activity 
supported by at least one resource, and one point was subtracted for 
each activity not so supported. Point totals are shown in the summary 
of "Raw Scores of Project Plausibility," in Appendix C-2. 
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Second, the study team assessed whether a11 proposed activities 
could be completed within the 18-roonth grant period. Activities judged 
doable were awarded one point; one point was subtracted for each 
activity that could not be completed In 18 months* 

Third, the study team determined whether all the proposed short- 
term outcomes could be achieved within the 18-month grant period. One 
point was assigned to those that could be achieved within the grant 
period; one point was subtracted for each that could not be achieved. 

Fourth, the study team determined whether the overall project 
design and outcomes (long-term and short-term) adequately 
reflected the two objectives required for the Cooperative Demonstration 
Program: 

1. Access to quality vocational education 
training; and 

2. Successful cooperation between public and 
private sectors. 

One point was assigned If the project design and/or long-term or short- 
term outcomes adequately reflected the required objectives for the 
Cooperative Demonstration Program. 

The design of the project was considered highly plausible If It 
met all four conditions. Projects that met three conditions were 
considered "plausible." Projects that met fewer than three criteria 
were considered "less plausible". 

Results of the Plausibility Assessment 

Description of the Plausibility Assessment . Thirty FY 1989 
projects were scored for project plausibility, and the results are 
shown In Table I 1-3. The findings are: 

■ Twenty-three of the 30 projects appeared to 
allocate sufficient resources to conduct 
their activities. 
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■ Twenty-nine projects appeared capable of 
completing the proposed activities within 18 
months. 

■ All thirty projects appeared capable of 
achieving the proposed short-term outcomes in 
the 18-month time frame. 



■ Twenty-one projects reflected the two 

objectives of the Cooperative Demonstration 
Program: access to training and a successful 
public/private partnership. 



Analysis of the Plausibility Assessment . Thirteen projects were 
scored as highly plausible, and 17 projects were scored as plausible 
(see Table 11-4). For an example, the application submitted by Alabama 
Aviation College met all four conditions. This project developed a 
training model and curriculum in aviation maintenance targeted toward 
minorities and females (underrepresented groups). Outcomes specifi- 
cally reflected the objectives of the program, i.e., access for under- 
represented groups and a successful public/private partnership. 
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C. Summary of the Evaluabilitv Assessment 



Thirty FY 1989 projects were assessed on the logic and 
plausibility of their project design as stated in their grant 
applications. On logic, nine were rated high, 16 medium, and five low. 
On plausibility, 13 were highly plausible, and 17 were p1ausib1e-no 
projects were "less plausible." 

A project with a good chance of succeeding should rank high on 
both logic and plausibility. By contrast, projects receiving a low 
rating in either logic or plausibility began with the disadvantage of 
little or poor planning. Table II-5 shows the logic and plausibility 
rankings of all 30 projects. Accounting for both logic and plausibi- 
lity, four projects were predicted to be highly successful, 21 projects 
were moderately successful, and five projects less successful. Table 
II-6 summarizes the predictions for project success. 

Comparing FY 1988 and FY 1989 Applications 

Table II-7 compares final rankings in project logic for FY 1988 
and FY 1989 applications-showing similar distribution of applications 
across categories. FY 1988 and FY 1989 applications differed on scores 
for project plausibility (Table II-8). More FY 1989 than FY 1988 
applications scored high in project plausibility. No FY 1989 projects 
scored low in project plausibility, while seven of the FY 1988 projects 
received low plausibility scores. 

In rankings of logic and plausibility used to predict project 
success, FY 1988 and FY 1989 projects showed similar rates for high 
success (Table II-9). However, FY 1989 projects were predicted to be 
potentially successful at twice the rate of FY 1988 projects. This was 
due, in part, to the fact that reviewers did not require FY 1989 
projects to mention explicitly a successful public/private partnership 
as one of its outcomes. If projects reflected the public/private 
partnership in its overall design and/or outcomes, grantees were 
awarded one point. 
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Table II-5 

SUMMARY OF PROJECT LOGIC AND PLAUSIBILITY 



Project Logic Project Plausibility 



Alabama Aviation College 


Low 


High 


Ben Hill Institute 


Medium 


Medium 


Bronx Community College 


Medium 


High 


Chattanooga State College 


Medium 


Medium 


Clackamus Community College 


High 


High 


Colunr)bia Basin College 


High 


Medium 


El Paso Community College 


Medium 


Medium 


Fox Valiey Tech 


Medium 


Medium 


Fresno City College 


Medium 


Medium 


Hampden County Consortium 


Medium 


High 


Home Builders Institute 


High 


Medium 


Howard Community College 


Low 


Medium 


Illinois Central College 


High 


High 


Illinois Eastern College 


Medium 


Medium 


Indian Hills College 


Low 


High 


John M. Patterson College 


Medium 


Medium 


LTV Steel Co. 


Low 


Medium 


Luzerne Community College 


High 


High 


Nebraska Labor Department 


Medium 


High 


Northampton Community College 


Msdium 


Medium 


North Clackamus Schools 


Medium 


Medium 
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Table I J -5. (Continued) 



Project Project Logic Project Plausibility 



Partners (PAVE) 


Medium 


High 


Research and Development Center 


High 


High 


Southwestern College District 


Medium 


High 


Valencia Community College (CIM) 


High 


Medium 


Valencia Community College (Health) 


High 


Medium 


Valencia Community College (Tele) 


Medium 


High 


Waubonsee Community College 


Low 


Medium 


West Virginia Department of Education 


Medium 


High 


West Virginia Northern College 


High 


Medium 
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Table II-6 



PREDICTION FOR PROJECT SUCCESS 




H = predicted to be highly successful 
{H = predicted to be potentially successful 
[]] = predictec to be less successful 
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Table I 1-7 

ANALYSIS OF PROJECT LOGIC: FY 1988 AND FY 1989 





High 


Hedium 


Low 


Total 1 


FY 


1988 


8 


10 


5 




23 1 


FY 


1989 


9 


16 


5 




30 



Table I 1-8 

ANALYSIS OF PROJECT PLAUSIBILITY: FY 1988 AND FY 1989 





High 


Medium 


Low 


Total 


FY 


1988 


4 


12 


7 


23 


FY 


1989 


13 


17 


0 


30 



Table 1 1-9 

RANKING FOR LOGIC AND PLAUSIBILITY: FY 1988 AND FY 1989 





Successful 


Potentially 
Successful 


Less 
Successful 


Total 


1 FY 


1988 


3 


10 


10 


23 


1 ''^ 


1989 


4 


21 


5 


30 1 
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As shown in Table 11-10, FY 1988 and FY 1989 projects were deemed 
logical at about the same rates, with the exception of condition rive 
(more linkages than gaps between short-term and long-term outcomes). 
Fewer FY 1989 than FY 1988 projects showed more linkages than gaps 
between short-term and long-term outcomes. 

In conclusion, the rate at which projects were predicted to be 
successful doubled from FY 1988 to FY 1989. This increase was linked, 
in part, to relaxing the condition that projects demonstrate 
public/private partnerships in scoring plausibility. Projects from 
both years were rated about the same with respect to logic, while FY 
1989 projects tended to score higher in plausibility. 



ReconOTendations to Strengthen Applications 

To strengthen grant applications-as well as the review process- 
the Office of Vocational and Adult Education could require a project 
flow diagram (see Figure II-3) as part of each applicant's "Plan of 
Operation." The following instructions could be given applicants: 



Instructions for Writing the Plan of 
Operation ; State the major objectives of 
your project. These should be entered under 
"Final Outcomes" in the fifth column on the 
flow diagram. Next, list the resources in 
the first column (including Federal and non- 
Federal) used to support the project. State 
dollar amounts. 

Under the activities column, list activities 
to be undertaken at the beginning of the 
project. Start each item with an action 
verb, e.g., "convene the advisory panel," 
"hire instructors," "assemble equipment." 
Then draw a line linking each resource with 
the appropriate activity. This shows which 
resources are allocated to each activity. 
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Table 11-10 
SCORING PROJECT LOGIC: FY 1988 AND FY 1989 



Conditions 


FY 1988 


FY 1989 1 


One or more long-term outcomes 


23 


30 1 


One or more short-term outcomes linked 
with long-term outcomes 


10 


11 1 


One or more activities linked with short- 
term outcomes 


21 


29 


More linkages than gaps between 
activities and short-term outcomes 


16 


23 


More linkages than gaps between short- 
term and long-term outcomes 


5 


2 
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In the third column, list short-term outcomes 
resulting from each activity; e^g., an 
outcome from "convene advisory panel" might 
be "draft recommendations of skills to be 
taught." Second short-term outcome might be 
"incorporate skills into training module." 

Draw a line between each activity resulting 
in a short-term outcome. Link each short- 
term outcome. Example: 



Resource Activity Short-term outcome 



Federal > hire > develop curriculum 

$300 , 000 i nstructors 



The diagram is complete if all short-term 
objectives lead to one of the long-term 
outcomes listed in column five. Draw a line 
linking short-term outcomes leading to 
separate long-term outcomes. If any short- 
term outcome does not directly relate to one 
of the long-term outcomes, it may not belong 
in the project. Base the "Plan of Operation" 
in the narrative section of the application 
using the flow diagram. Don't foroet to 
include the flow diagram with the Plan of 
Operation . 



Other Federal grant programs, e.g., demonstration grants for 
preventing alcohol and other drug abuse, have required applicants to 
include logic models as a formal part of the application. The 
instructions given to their applicants are as follows: 



A logic model should underlie the conceptual 
development and preparation of the proposal. The 
logic model is a conceptual framework that links 
(1) basic assumptions about risk/protective 
factors, (2) mechanisms of intervention and (3) 
outcomes. In this model, the application: 

- specifies the risk factors and/or 
resiliency/protective factors at the 
individual, parent and family, peer 
group, school or neighborhood levels 
being addressed in the proposal; 
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resiliency/protective factors to the 
use and abuse of alcohol and other 
drugs by youth; literature documen- 
tation; 

- relates the proposed strategies or 
interventions to specific risk/ 
protective factors, that is, 
explains why these interventions 
should help reduce these risk 
factors or enhance these resiliency 
factors; and 

- identifies the measurable/observable 
outcomes that can logically be 
anticipated from the interventions. 



OVAE may wish to include a similar requirement in future demonstration 
regulations* 
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Section III 
PROJECT IMPLEMENTATION 



III-l 



III. PROJECT I MPLEMENTATION 



A. Framework for Studying Implementation 



The analysis of project implementation was based on a plan 
developed early in the study but later revised. In the original plan, 
the study team concentrated on the following set of quest ivons: 



Project Administration: 

■ Did the project establish the administrative 
infrastructure necessary for its activities, 
i.e., did the project hire staff, obtain 
space, schedule services, engage an 
evaluator, etc.? To what extent was the 
project infrastructure already in existence 
when the grant began? 

■ What kinds of partnerships were established 
and what problems arose in creating them? 
Was public-private cooperation established, 
and what other partnerships arose (with 
support service providers, service delivery 
areas, etc.)? 

■ What did projects do to prepare for training 
or other direct services, e.g., skill 
identification, curriculum development, 
licensure, coordination with other providers, 
or staff training? 

■ What was done to recruit students? In 
particular, what type of students did 
projects seek to attract » and how did the 
projects go about doing it? Were there 
particular efforts to attract special 
populations, and, if so, which populations? 

■ What was done to continue the project beyond 
the 18 months of the Federal grant? If the 
project was continued, were the activities 
successful? 



Project Content and Intermediate Outcomes : 

■ What kinds and amounts of training or other 
direct services were provided to students 
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through the grants? How many were trained? 
How many got jobs? 

■ How did projects Identify needed job skills 
an<j how was the Information used? Project 
proposals suggested substantial amounts of 
tiine for skill Identlflcatlon-hence the 
evaluation sought to determine the use of the 
skill Information, 

■ What curricula were developed and how were 
they used? As with skill Identification, 
curriculum development was emphasized In 
grant applications. 

■ What linkages were developed with other 
public sector agencies or community-based 
organizations? With what effects? To what 
extent did other agencies aid In recruitment, 
training, placement, etc.? 



Extent and Kinds of Innovation : 

■ To what extent did the grants provide 
opportunities for new directions and 
experimentation? 



After analyzing the FY 1988 grantees and incorporating the 
suggestions of the advisory panel and Federal officials, the study team 
revised the evaluation plan. Although efforts to characterize the 
projects along the first two dimensions outlined above were retained, 
the Implementation analysis was expanded to focus on five Issues: 



■ Nature of the training; 

■ Type and Intensity of public-private 
cooperation; 

■ Meaning of "high technology" within the 
projects; 

■ Extent to which the projects were 
operationally successful; and 

■ "Exportablllty" to others of what was 
developed or learned at each site. 
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The final item, exportabil ity, substituted for the examination of 
innovation that was originally planned. The Federal office 
administering the program argued that it was not a condition of this 
demonstration program that projects be new, innovative, or 
experimental. Hence, assessing projects on the extent of their 
innovativeness would be unfair. Each of the five issues is briefly 
discussed below. 

The Nature of Training . The evaluation of FY 1988 grantees showed 
that the types and amounts of training delivered under the program 
differed considerably among the projects. The range of training 
activities was impressive— from half-day workshops for business 
managers on the virtues of a particular piece of equipment to two-year 
technical degree programs for nontraditional students. The range also 
meant, however, that it was hard to discern the precise nature of the 
Federal demonstration, to understand if there were sufficient 
commonalities among the projects to describe them jointly, and to 
conduct the cost-benefit analysis which was a mandated part of the 
evaluation. 

There was also a specific question raised in the Year One report 
about the efficacy of customized training as a part of the 
demonstration effort. Three of the eight Year One high-tech grantees 
visited by the evaluation team were customized training projects. The 
grantees— usually community colleges— provided highly specific training 
to the employees of individual companies. Often the training was 
designed to enable the employees to use a piece of specialized 
equipment or software, although it might also be aimed at teaching more 
generic occupational or basic skills. The regulations of the 
Cooperative Demonstration Program note that all projects assisted under 
the program must be capable of wide replication by service providers— 
an outcome that seems questionable for many customized training 
projects. Questions were raised in the Year One report about both the 
overall desirability of narrow training under Federal sponsorship, as 
well as about the specific priorities that could be introduced to 
ensure that some broader public interest was served. 
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The advisory panel also raised questions about the advisability of 
supporting demonstration projects that provided students with short- 
term training aimed at specific, entry-level jobs (regardless of 
whether It was customized for a particular employer). This form of 
training was offered by many of the Year One grantees visited. The 
panelists argued that demonstrations should be concerned with how to 
Increase trainee skills and employment options, not just with the 
demands of specific entry-level jobs. There was considerable interest 
in whether projects supported through the program were engaged in 
public-private cooperative ventures to deliver both job-specific and 
more general skill training and what could be learned from such 
ventures. 

The Type and Intensity of Public-Private Cooperation . Based on 
Year One findings, the evaluation team identified a variety of public- 
private relationships: 

■ The private sector partner was a customer of 
the project-the project provides training for 
employees of the company; 

I The partner was a member of an advisory 
committee or an Informal board; 

■ The partner was a supplier of resources such 
as equipment, and/or jobs at project 
completion; 

■ The partner was an active participant in 
delivery of instruction or other services; or 

■ The partner initiated the project and 
approached the grantee for assistance. 

Among Year One grantees, most of the projects visited had 
relationships with private sector partners in which the partner played 
a customer or advisor role, sometimes also contributing equipment for 
training. A few projects did not have private sector businesses as 
partners. In only one of the eight grantees visited did the evaluation 
team see a project in which a private sector business had participated 
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actively in both the design of the project and in service delivery. As 
with the search for interesting training models, then, the desire in 
Year Two was to identify partnerships that might be breaking new 
ground, i.e., that might provide models of public-private cooperation 
for high-tech training. 

Defining Hi ah Technology . Based on Year One findings, the meaning 
of high technology, like the meaning of public-private cooperation, 
varied considerably. As discussed in Section I, the definition 
included in the law and regulations was quite broad. Based on Year One 
findings, the study team characterized the actual high tech elements of 
the projects the study team visited first in terms of the field for 
which students were preparing. If the field was not high tech, the 
study team then focused on the specific training received. Four 
definitions of training for high technology emerged: 



I Training was designed to prepare students for 
jobs in fields generally considered high-tech 
because they manufactured (or serviced) high 
technology products. The training itself may 
have been sophisticated or basic, and have 
led to upgrading skills of current workers or 
to entry-level jobs. 

I Training enabled students to use high-tech 
equipment even though the field in which they 
worked (or sought work) was not considered 
high tech. For example, one project taught 
computer assisted design (CAD) or the use of 
sophisticated diagnostic equipment to repair 
automobiles. 

I Training taught students to use computers, 
irrespective of the field for which training 
occurred. For example, some projects 
prepared students to be secretaries by 
teaching word processing skills. 

I Training was provided in basic skills as 
preparation for later occupational training 
in a high-tech field. The grant supported 
the basic skill phase of instruction with the 
understanding that the students who improved 
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their basic skills would enter training that 
met one of the three previous definitions. 

In short, not all projects shared a conmon understanding of what was 
meant by high-tech training. To identify the range of projects, 
especially those that were training students in ways that appeared to 
be "state of the art," the second year of the evaluation focused on 
understanding the high-tech elements of the projects studied. 

Operation and Integration of the Demo nstration within the 
Institution . In Year One there were a number of projects that 
experienced delays in start-up and appeared to be in danger of 
terminating with the end of the Federal grant. The study team 
identified timing issues that appeared to affect start-up, but reasoned 
that projects with stronger ties to the institutions Tin which they were 
located might have a better chance to start promptly and continue after 
the end of the grant period. They might also be better able to draw 
upon the expertise of regular teaching staff and use facilities 
effectively in program development and delivery. This section looks at 
this and other problems that arose in implementing project designs. 

ExDortabilitv of Project Design. A>:tivities. and Products . The 
Cooperative Demonstration Program was intended to demonstrate models of 
public-private cooperation in the delivery of training. As stated in 
the regulation, all projects were to be "(1) Of direct service to the 
individuals enrolled; and (2) Capable of wide replication by service 
providers" (CFR 34, Ch. IV (7-1-88 Edition), Part 412, Subpart B). The 
program was to produce 

model projects providing improved access to quality 
vocational education for [special populations] and 
...examples of successful cooperation between the 
private sector... and public agencies in vocational 
education... 

The way in which the study team chose to characterize the demonstration 
element of the Cooperative Demonstration Program was exportability, 
i.e., that which can be taken from the projects that have been 
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supported and applied elsewhere. In other terms, what can a service 
provider interested in effective partnership use or apply from what has 
been produced as a result of the Cooperative Demonstration Program? 

Some projects were clearly demonstrations by design. These were 
projects aimed at development of a specific product, usually a 
curriculum, that can be applied by others. They used the grant to 
develop the curriculum and conduct a formative evaluation (pilot test) 
or they used the grant to test the replicability of the curriculum in 
additional institutions or with new populations. These projects were 
usually characterized by formal evaluation designs that accompanied 
project (curriculum) development and implementation. In Year One, only 
a very few of the projects were of this type. 

More commonly, projects were aimed at adding dimensions to, or 
improving the delivery of, training at the grantee institution. While 
that was certainly a worthwhile goal from the institution's 
perspective, it does not necessarily translate into a demonstration 
unless the process of institutional "capacity building" yields two 
things. First, it must provide evidence of effectiveness-i.e., that 
the procedures undertaken did, in fact, build the institution's 
offerings or improve the delivery of instruction in measurable ways. 
Second, it must yield a guide or other device that shows others how 
they can accomplish the same changes using the same procedure s-i .e., it 
must be made capable of replication. The study team sought to 
determine which, among the FY 1989 projects, held promise as models 
capable of replication. 

The implementation analysis in this section is organized around 
these five issues. If the reader would like additional information on 
project administration, content, and intermediate outcomes, please 
refer to the report on the overall findings of the implementation 
survey in Appendix F. Additional information, including more detailed 
grantee-by grantee-descriptions of project activities, partners, 
clientele, staffing, and dissemination can also be found in the matrix 
of projects at the end of this section. 
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B. Procedures for Studying Implementation 



The implementation analysis included four data collection steps. 
The first two steps were carried out with FY 1988 grantees. Shortly 
after the evaluation began (March 1990) the evaluation staff refined a 
set of implementation issues outlined in its technical proposal, 
selected nine Year One grantees for further study (based on a 
preliminary evaluability assessment), and conducted a telephone survey 
of the nine grantees (April, 1990). Immediately after the telephone 
survey, the team conducted visits to eight of the nine grantees. All 
the visits were completed by the end of June, 1990, when most of the 
Year One projects ended. 

These two data collection steps (telephone survey and visits to 
eight projects) yielded data that were the basis for the implementation 
portion of the Year One report. They also provided the evaluation team 
with the opportunity to reflect on the original design and make some 
minor modifications. These changes were discussed earlier in this 
chapter . 

The two data collection efforts in Year Two were considerably more 
ambitious. After 0MB clearance, a systematic mail survey was mounted 
with questionnaires distributed to all 30 grantees. Despite initial 
mailings by the OVAE program managers ar^ extensive telephone follow 
ups, only 26 grantees (87 percent) retur;:ad questionnaires. (Results 
are reported for 27 grantees, however, because one grant yielded two 
sub-grants with unrelated projects.) Names of respondents are included 
in Appendix F, the report of survey findings. The survey was followed 
by two-day visits to 19 of the 30 FY 1989 grantees. 

The site visit teams used a similar protocol to the one used in 
Year One, although a few changes were made during the 0MB clearance 
procedure. As in the first year, the study team prepared site visit 
reports. The site visit information was then condensed into three- to 
four-page descriptions (see Appendix E). Based on the descriptions and 
discussions with site visit teams, the study team created the data 
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displays at the end of this section summarizing the qualitative data 
obtained from each site. 

The Year Two data collection efforts form the basis for the 
results reported in this chapter. Specific information from the mail 
survey is referenced in the text. For some issues, the study team 
relied more heavily on site visit information. For others, the study 
team relied primarily on information derived from the survey. The 
study team tried, whenever possible, to supplement one with the other. 
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C. Haior Findings from the Implementation Study 



K The Nature and Breadth of Training 

There are several dimensions along which the training efforts of 
FY 1989 projects way be discussed* One dimension is simply the extent 
of training, i*e*, the number of hours, weeks, or months that students 
typically participate. A second dimension is the content of the 
training-is it detailed, job- or equipment- specific instruction, an 
overview or introduction to a field with generic as well as more 
specialized information, or is it refresher basic Skills instruction in 
preparation for occupational ly-specific instruction later. Individual 
Cooperative Demonstration projects reflect all of these kinds of 
training, and several projects combine several different kinds of 
training. 

The amount and content of training is also related to other 
factors, the most important of which is probably the clientele. 
Specialized instruction may be more appropriately delivered to some 
clients than others, notably to persons who are already engaged in 
occupations akin to the area of instruction. 

Amount of Training . The FY 1989 projects that were visited 
offered relatively short educational programs. Although the designs of 
the various projects ranged from workshops of less than a day to two- 
year programs yielding associate degrees, the majority of projects 
developed short-term offerings* Many of the projects provided training 
to more than one clientele or for more than one type of job. with the 
amounts of time varying by clientele and position. 

The shortest training efforts were essentially workshops, i.e., 
programs of one day or less. These were commonly offered to business 
owners or employees, often to demonstrate new technologies or 
equipment. For example, one project provided weekly two-hour 
demonstrations of computer integrated manufacturing (CIM). Employers 
then signed up for short-term training in computer assisted drafting 
(CAD)» computer assisted manufacturing (CAM) and other topics with 
training usually lasting one day. Another project offered four-hour 
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workshops on disk operating systems (DOS), CAD and CIM. A third 
project introduced large numbers of employees in one company to a new 
data collection system, training them in three-hour sessions before 
work, followed by two weeks of on-the-job training (OJT). 

Several projects were designed to offer 100 to 200 hours of 
instruction spread over a six-month period. For example, one project 
taught students a computer language, providing 108 hours over six 
months. Another project offered a series of semester-length courses in 
CAD and programmable logic computers. Two projects offered entry-level 
training programs for health-care technicians of less than six months 
duration. One project provided students a minimum of 12 hours a week 
of training in business and office skills, with students remaining 
until placed in jobs-typically about four months. 

Another set of projects offered training lasting a year or less. 
These projects tended to offer a license or certificate at completion. 
One project created an 11-month program to train respiratory therapy 
technicians, while a second health-related project trained students as 
technicians in a variety of specialties through one-year programs. 
Another project established a six- (later eight) week overview and 
nine-month certificate program in precision metal fabrication and screw 
machine technology. One project trained disadvantaged adults in 
desktop publishing during a 400-hour program, while another developed a 
one-year program in software applications. 

A final set of training projects was designed as two-year programs 
leading to technical degrees or licenses. For example, one project 
designed a two-year training program in hazardous waste management. 
Another was initially designed as an 18-month training effort for 
radiology technicians but will be expanded to 24 months in the future. 
The one FY 1989 project aimed at high school students created a two- 
year occupational "clusters" program with internships. Two grants 
supported institutional capacity-building efforts, with one school 
developing an associate program in advanced technology and the other 
revising an existing associate-level industrial technology program. 
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Projects aimed primarily at curriculum or product development are 
harder to characterize using the time dimension. One project developed 
a curriculum to train entry-level employees in semiconductor 
manufacturing over approximately 283 hours. The training portion of 
this project was intended to pilot test the product. Another project 
developed an 18-hour program for master electricians to enable them to 
install home automation systems. Three projects developed one or more 
videodiscs to be used in conjunction with ongoing training programs. 

The relatively short duration of training observed in the 19 site 
visits was reflected by the findings of the survey of FY 1989 grantees. 
Grantees reported that 68 percent of participants received between one 
and 100 hours of training, while 11 percent received 101 to 250 hours. 
Approximately 14 percent of participants received 1,000 hours of 
training or more. 

There is a major caveat to note with respect to the hour totals, 
however-one that suggests that the actual number of grant-supported 
hours of training may have been lower than these numbers reflect. Not 
all projects reported total instructional hours received through the 
grant in the same manner. The questionnaire instructions asked for 
"hours of instruction... from the project during the grant period," but 
it was likely that at least some projects included total instructional 
hours received by participants, whether or not the training was project 
supported. For example, a project may have provided revised or 
additional services to an ongoing course or degree program. Some sites 
reported the total number of hours required to complete that course or 
degree program as the instructional hours received under the grant. 

Training Content . As with the duration of training, there was 
considerable range in content areas. Most of the projects that 
received site visits offered training in three broad 
areas-manufacturing, health, or business/office. Projects in 
manufacturing appear to have been focused on both entry-level and more 
advanced skills, while the health and business/office projects were 
focused on the development of entry-level skills. 
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Projects offering manufacturing-oriented training tended to train 
participants to use similar types of high technology equipment. At 
least five of the projects taught students computer-assisted 
manufacturing applications including the use of CAD, CAM, computer 
numerical controlled (CNC) machining or programmable logic controllers 
(PLCs). An additional project taught the application of a specialized 
computerized manufacturing system for a particular plant. These 
projects tended to be short-term. Some began with demonstrations of 
the use of the equipment, followed by short classes for persons seeking 
to use the systems. Almost all the students in these projects were 
current employees of companies. The one exception was a two-year 
program in the use of CNCs aimed at women seeking work in manufacturing 
(although the majority of students in the program were men). 

A few manufacturing projects receiving site visits did not fit the 
short-term/CAD-CAM model. These included a 48-month apprenticeship 
program for Electronic Instrument Repair Technicians 'EIRT), an ongoing 
program in which the Cooperative Demonstration grant supported an 18- 
month portion; and the nine-month certificate programs in precision 
metal fabrication and screw machine technology. The model may also not 
fit precisely a project that trained pipefitters and millwrights to use 
a new, automated equipment system at a steel mill. The EIRT program 
and pipefitter/millwright programs were two of the most customized 
training efforts, geared to the specific machinery of the partner 
companies. Another project offered a wide range of entry-level 
programs for disadvantaged students, including a 254-hour basic 
machining course. Finally, one manufacturing project revised and 
expanded the offerings of an industrial technology program at a 
community college. 

The health projects were generally of longer duration and were 
geared to entry-level technician positions. For example, one project 
trained students as licensed practical nurses (LPNs) and alcohol /drug 
counselors. It provided a year of specialized training and OJT 
combined with remedial basic skills. Another project offered both a 
5.5-month training program for pharmacy technicians and an 11-month 
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training program for respiratory therapy technicians. The longest such 
program for adults was 18 months and trained participants to be 
radiology technicians. All of these projects had a clinical component. 

There were two health projects that did not reflect the health 
technician training profile. One developed interactive videodiscs that 
taught viewers to use volumetric pumps. It was designed to be used by 
students already familiar with intravenous fluid delivery and was 
tested in a refresher course for returning nurses. The other project 
cut across the health and business sectors, training disadvantaged 
persons for entry-level work in the health field, although much of the 
training was for clerical positions (i.e., medical transcription, 
medical records apprentice, medical unit clerk). Students in this 
project received an average of 120 to 140 hf^"rs of instruction. 

In general, business/office projects that received site visits 
tended to focus on entry-level skill training, although the hours of 
training varied widely. Two projects were extensions of Job Training 
Partnership Act (JTPA) training programs. One provided a 400-hour 
training program in desktop publishing (as well as other, non-business 
offerings), while the other offered a set of 40 self-paced office 
technology courses. Students with prior typing skills also received 
unpaid work experience, and it was estimated that the typical student 
spent about four months in the program before finding employment. The 
high school office systems occupational cluster program was two years 
in length, with an internship of four to six weeks. Finally, one 
business grantee designed several different options, including a one- 
year program leading to a certificate in microcomputer software 
applications. 

There were a few projects that included training in fields other 
than manufacturing, business, and health. As already noted, one 
project developed an 18-hour curriculum to enable master electricians 
to install home automation systems. In addition, one of the JTPA 
programs that offered a wide range of courses offered a 254-hour 
program in automobile repair. Two projects were focused on computer- 
related training, with one developing a six-month program in ADA 
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computer language and another "packaging" a number of courses to create 
a two-year advanced technology degree. 

Only a few of the projects that received site visits spent large 
amounts of instructional time on non-occupational training. The most 
common form was remedial basic skills instruction, which was provided 
in health projects and others geared to disadvantaged students. For 
example, the project providing training for medical /clerical entry- 
level work also provided students with a self-paced, computer-based, 
remedial skills program. The project to train LPNs and drug/alcohol 
counselors reported that it provided a total of 2,000 hours of remedial 
education. Also providing basic skills instruction were the 
apprenticeship project for EIRTs and the project to train entry-level 
workers in manufacture of precision metals. It was likely that more 
advanced academic skill instruction was included in the various 
certificate and associate level programs. 

The relationship between industry and duration of training was 
reflected in findings from the survey of grantees. Respondents 
identified the industries in which they trained, and each response was 
then compared with the hours of training reported for participants in 
that project. Most of the students (82 percent) in projects that 
indicated manufacturing as an area of training received less than 100 
hours of instruction (see Table III-l). This compares with 43 percent 
of students in business service projects and six percent in health care 
projects. Overall, health care projects delivered the largest number 
of hours of training, with most students (80 percent) receiving 250 
hours or more. 

In addition to occupational ly specific training, a number of 
projects offered basic skills or other related instruction. According 
to questionnaire responses, the majority of the projects (16 of 27) 
provided non-occupationally-speclfic training. The most common forms 
of non-occupationally-specific training were provision of employabil ity 
skills (13 projects), provision of basic academic skills, (11 
projects), and provision of advanced academic skills (9 projects). 
Interestingly, there was little relationship between the field of 
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training or hours of training and the likelihood of non-occupational 
instruction, but projects geared to disadvantaged students were more 
likely to provide non-occupational training. 

Clientele . Observations derived from site visits were reflected 
in survey findings as well. According to the survey, projects 
providing training in manufacturing were the most likely to indicate 
current employees as their primary clientele type. As shown in Table 
I I 1-2, ten of 17 suci projects indicated that current employees were 
their primary clients. Projects providing training in multiple 
industries also commonly indicated that they served existing employees 
(four of seven projects). Projects providing health care training were 
more likely to indicate that they served adults seeking work in a field 
or company (three of seven). Only one project providing health care 
indicated that it served current employees of companies as its main 
clients. 

Projects offering training in business services were somewhat more 
likely than others to indicate that they served disadvantaged adults 
(two of five projects). Projects training for manufacturing and 
multiple industries were also more likely to be training persons 
employed full time (see Table III-3). Although information on 
employment status of clientele was not available for all such projects, 
of the 16 projects training students in manufacturing for which data 
exist, seven indicated that 76 to 100 percent of students were employed 
full time, and an additional three indicated full time employment by 51 
to 75 percent of students. In contrast, none of the six projects 
providing training in health care for which information was available 
indicated the 76 to 100 percent of students were working full time, and 
only one indicated that 51 to 75 percent were working full time. 

Summary . Overall, projects provided relatively short-term 
training. Survey results indicated that most participants received 
training of fewer than 100 hours. Projects included in the site visits 
can be grouped loosely into the following three categories: 
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■ Short-term, skill-specific instruction for 
current employees of companies or for persons 
already familiar with the field of training. 
This approach was most common among manufac- 
turing projects, but it could also be seen in 
the refresher nursing course and in business 
projects teaching specific software 
applications, as well as home automation 
installation. These projects rarely provided 
formal credentials. Some instruction was 
customized, tailored to the needs of specific 
employers. 

■ Medium-term, entry-level technician training. 
These were programs in which training ranged 
from a few months to a year, often aimed at 
helping unemployed or otherwise disadvantaged 
persons find jobs in a field. These programs 
cut across the industries but were quite 
common in business. The three programs run 
by JTPA-supported agencies fell within this 
category, one of which provided training in 
several fields. Some provided certificates. 

■ Longer--term programs aimed at two-year 
certification or degrees (associate degree, 
license). Some of these projects were aimed 
at institutional capacity-building and 
regular students; e.g., creating new or 
revising existing offerings within depart- 
ments of industrial technology or business/ 
computers in community colleges. One was an 
apprenticeship training project, and one an 
occupational project for high school 
students. 



Not all 19 projects that received site visits fit neatly into one 
model, and a very few may not fit at all. But most could be identified 
within one or more of these groupings, which provided some sense of the 
range of programs developed and training delivered under the Coopera- 
tive Demonstration Program. 
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2. Type and Intensity of Public/ Private Cooperation 

The Cooperative Demonstration Program was designed to encourage 
cooperation between educational institutions and the private sector. 
As the program* s grants announcement noted: 

High technology training can be conducted most 
effectively with the active involvement and 
cooperation of the private sector. Effective 
partnerships between the private sector and public 
agencies in vocational education are an important 
aspect of the Cooperative Demonstration Program... 

The announcement anticipated that the partnerships established by the 
project would provide models of effective cooperation. To understand 
the nature and extent of cooperation, the FY 1989 respondents were 
asked numerous questions about their projects* public-private 
partnerships. The following discussion presents the survey results 
first, followed by findings from the site visits. 

Survey Findings . Most projects established relationships with 
einployers. Of the 27 respondents, 23 indicated some involvement of 
employers in the planning or administration of the project or in 
providing services. Nineteen projects indicated multiple types of 
partners, including schools or universities (institutions that may 
serve the ••private sector" cooperative role or may be involved in the 
project in some other manner) and community-based organizations (ten 
projects) . 

Although employers were involved in 23 projects, not all those 
projects indicated that private businesses were the most important 
partner organizations. Only slightly more than half the projects (14 
of 26) indicated that the most important partner was a private 
business. Of th« remaining projects, five indicated that the most 
important partner was a trade association or consortium, four indicated 
an educational institution, and three indicated a non-educational 
public agency. For the grantees that had second partners ♦ these 
partners were even less likely to be private businesses (nine of 20) 
and more likely to be trade associations (four of 20). 
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Most grantees formed partnerships with only a few organizations, 
but some established multiple partnerships. Six projects had one 
private sector partner, and 16 projects had four partners or fewer, but 
seven projects indicated that they had ten or more partners. Given 
this range in the number of partners, it was likely that the roles of 
partners differed considerably across the projects. 

In fact, differences in partner contributions were considerable. 
When asked to rank. in importance a list of possible activities for the 
most important partner organization, there was little consensus among 
the grantees. The most commonly selected first choice activity for the 
partner organization was that it provided equipment for training, but 
that choice was selected by only seven grantees. Five grantees 
indicated that the partner's main contribution was identifying job 
skills, while four indicated that the partner recruited students for 
training. Three indicated that the partner served on a project 
advisory committee and two indicated that the partner supplied 
instructors. In short, there was considerable variability in first 
partner contribution-a variability that was sustained when the first 
through third ranked activities were added together. The same findings 
held true for second partners and, in fact, the variability in 
contribution increased. 

The differences in activities may be, in part, a function of the 
nature of the partner relationships. Based on categories developed 
from site visits to first year grantees, the evaluation team identified 
several ways to characterize or summarize the relationships between 
grantees and employers. Each FY 1989 grantee was asked to pick the 
choice that best characterized its relationship with its first partner. 
The majority of grantees selected one of the following two choices: 

■ The partner was a customer of tht project, 
e.g. the project provided customized training 
to the partners* employees (selected by nine 
grantees); or 
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I The partner shared actively in the delivery 
of instruction and services (selected by nine 
grantees) . 

No other choice was selected by more than three grantees (see Table 
III-4). 

Projects with multiple partners were more likely to report that 
the partners were customers, while projects with few partners used 
partners in more ways. Of the nine projects selecting "customer" as 
the main relationship, five also reported seven or more partners. In 
contrast, eight of the nine projects indicating that partners shared in 
instruction reported six or fewer partners. In addition, projects with 
six or fewer partners reported other uses of partners (advisory 
committee member, resource provider, initiator of project) while none 
of the projects with seven or more partners selected any of these 
categories. 

In most of the projects, the grant did not result in establishment 
of a new primary partnership. Nineteen of 27 grantees reported that 
the relationship between the grantee institution and the partner 
organization was not new, but all 19 indicated that the relationship 
had been strengthened as a result of the project. The eight projects 
that did establish new relationships with the first partner 
organization were more likely to have encountered problems. Three of 
six projects that reported problems with their first partner had new 
partners, and two of these partnerships were dissolved b the end of 
the grant. 

Grantees were slightly more likely to have established new 
partnerships with their iecond partners. Eight of 21 second partners 
were new, which appeared to reflect a wider range of relationship 
types. Considerably fewer second partners were project customers 
(two), though a sizeable number (eight) shared in delivery of 
instruction and services. This difference between first and second 
partners may huve occurred because second partners were more likely to 
be trade associations or other educational institutions rather than 
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employers. Second partners were somewhat more likely to serve 
primarily as members of advisory committees. 

Site Visit Findings . Among FY 1989 grantees that received site 
visits, partners and their roles ranged widely. In a majority of 
cases, grantee institutions formed partnerships with a small number of 
private companies (or associations) that performed roles such as 
serving on advisory committees, loaning or donating equipment or 
facilities, or helping to identify the skills needed in the jobs for 
which training was provided. This was the case, for example, for the 
college that created a CNC training program aimed at attracting women, 
the college that established an advanced technology center that 
delivered courses to employees of local companies, and the county 
vocational center that taught ADA computer language. But in several 
projects, the study team saw different types of relationships, related 
directly to the way in which the project was organized and the services 
del ivered. 

The first type of cooperation was intensive customized training. 
In two projects, the private employer was either the initiator of the 
project or played a central role in design and service delivery. Both 
of these were customized training projects in which the employers were 
steel mills. In one case, the company had already established an 
apprenticeship program for repair technicians (EIRT). It had paid for 
curriculum development by an instructor at a nearby college and needed 
a process control lab for instruction. The company approached the 
college to apply for Cooperative Demonstration funding. As the grant 
could not pay for equipment, it was used to subsidize training and the 
steel company paid for the lab equipment. In the other case, a college 
and a steel company worked jointly to develop training that enabled 
over 1,500 workers to implement new, high-tech milling equipment. The 
company paid for curriculum development and supplied training space, 
OJT, user support teams, plant floor support teams, and vacation 
reimbursement for the time workers devoted to training before work 
hours. 
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Other projects reflected the customized training model in somewhat 
less intensive form. Several projects began by trying to enlist 
students through convincing employers that training could benefit them 
and their workers- Projects staff then worked with individual 
employers to develop instruction tailored to the needs of their 
companies, and employees received short-term training. In these cases 
the training was not quite as customized (CAD, CAM, etcO- Most of 
these projects hoped to continue after the grant by generating fees 
from tne businesses for the same or similar forms of instruction. 

The second type of relationship was recruitment bv the grantee and 
referral to the partner for training . In these cases the grantees-some 
of which were administered by agencies that receive ongoing support 
from JTPA-were community-based agencies (or other entities) that 
recruited participants. They might also assess the students' skills 
before referring them to another institution for training. They viewed 
their primary partners as being the colleges, technical institutes, or 
other institutions (such as hospitals) that provided the actual 
training. These grantees tended to view the private sector primarily 
as potential employers. Potential employers such as welfare agencies, 
might also refer persons to the grantees. These projects tended not to 
have advisory committees of employers, but stayed in contact through 
efforts to place students in jobs. One such project described itself 
as "managing** a partnership of training institutions and potential 
employers. 

The third type of relationship occurred in projects focused on 
curriculum development . The projects used schools or other 
instructional settings to "test" their offerings, and private sector 
partners (businesses or associations of businesses) as ongoing 
reviewers of materials. Some also used the employers to obtain initial 
information on skills that was then incorporated into curriculum 
development. The projects aimed at instruction in semiconductor 
manufacture manufacture of precision metals, use of volumetric pumps, 
development of high school occupational clusters, and installation of 
home automat-ion operated in this manner. Because the high school and 
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volumetric pump projects also Involved internships, the employers also 
supervised the students who were piloting the approach and provided 
feedback. 

Summary . In many projects, then, the nature and organization of 
project activity was linked to the type of partnership that arose. 
Among projects included in site visits, intensities of partnerships 
varied from the two projects where a single employer played a critical 
role in service development and delivery, to projects in which 
employers might do little more than attend meetings of an advisory 
committee. Overall, intensity of partnership may be less significant 
than number of partners in determining partner role. Survey results 
show that projects with multiple partners were considerably more likely 
to report that partners were customers, while projects with few 
partners used partners in more ways, including active sharing of 
Instruction. 

3. The Meaning of High Technology 

The projects also reflected different interpretations of the 
Congressional mandate that projects Involve high technology. As 
indicated previously, the evaluation identified four basic ways in 
which FY 1988 projects Incorporated high technology in developing their 
programs and delivering training. Projects were high-tech in that: 

I Training was designed to prepare students for 
jobs in fields that manufactured high-tech 
products or serviced high-tech equipment; 

I Training was designed to enable students to 
use high-tech equipment or products even 
though the field in which the equipment was 
used was not generally considered high-tech; 

■ Training was conducted on high-tech 
equipment, such as computers, CAD, or CIM 
equipment; or 

■ Training was offered in basic skills as 
preparation for specific occupational 
training in a high-tech field. 
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Survey Findings . FY 1989 survey respondents were asked to select 
all the high-tech definitions above that were appropriate to their 
projects. Most respondents selected more than one response to describe 
their projects* high-tech elements. Therefore, the study team created 
one response category for each project for further analysis: 

■ Training for high-tech field-11 respondents; 

■ Training for high-tech equipment in non-high- 
tech fieldHiine respondents; 

■ Training for both high-tech field and for 
using high-tech equipment in a non-high-tech 
field-three respondents; 

■ Training on high-tech equipment-one 
respondent; 

■ Basic skills training to prepare for further 
training in a high-tech field-two 
respondents; or 

■ Basic skills training utilizing high-tech 
equipment in preparation for further training 
in a high-tech field-one respondent. 

Using these categories the study team first observed that most of 
the projects (23) defined "high-tech" in terms of the occupation or 
equipment for which training was provided ♦ Of the 23, about half (11) 
were explicitly preparing students for immediate or specific work in 
high-tech fields. Most of the others (nine) were preparing students to 
use high-tech equipment or products in a non-high-tech field, with the 
rest doing both. If these answers were indicative, the remaining four 
projects were not preparing students for explicit high-tech applica- 
tions, but rather defined their projects* high-tech element in terms of 
the equipment used in training or the students* long-range occupational 
goal. 

Second, the study team examined the extent to which the 23 
projects that defined their high-tech focus in terms of occupation or 
field application also used high-tech equipment or provided basic 
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skills instruction as part of their activity* The study team found 
that of the projects that prepared students for work in high-tech 
fields, the majority (eight of 11) used high-tech equipment in training 
(see Table III-5)» Among the projects preparing students to use high- 
tech equipment in non-high-tech fields, however, only a slim majority 
(five of nine) conducted training on high-tech equipment. Since these 
projects were explicitly preparing students to use high-tech equipment 
or products on the job, the lack of training on such equipment in a 
majority of them was surprising. It suggests that the training may 
have been quite limited in scope. 

The projects preparing students for using high-tech equipment in 
non-high-tech settings were also more likely nfit to offer basic skills 
instruction. Only two of the nine such projects indicated basic skills 
instruction, compared with six of the 11 projects preparing students 
for jobs in high-tech fields. This finding also suggests that the 
training for use of high-tech equipment in non-high-tech fields may 
have been rather limited or narrow in scope. 

Projects that included manufacturing as an area of training were 
considerably more likely than others to prepare students for work in 
high-tech fields. Eleven of 17 such projects (65 percent) prepared 
students for high-tech fields (see Table III-6). Projects described as 
training for multiple industries also were weighted toward preparation 
for high-tech fields, with four of seven selecting this choice. This 
compares with only one of seven projects that provided preparation for 
health care jobs. None of the four projects that included training for 
the transportation industry indicated it was training for a high-tech 
fields and business service projects were about equally divided between 
high-tech and non-high-tech fields. 

Site Visit Findings , The site visit results parallel those of the 
survey, although sites that were visited tended to do more training for 
use of high-tech equipment. Most of the projects were preparing 
students to use high-tech equipment on the job, whether or not the 
field for which training was conducted was itself high-tech (see matrix 
of pro,iftcts at the end of this chapter). Many of the projects also 
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used high-tech equipment, especially computers, in the course of 
instruction, but only one project appeared to meet that high-tech 
criterion alone. 

Initial concerns that projects might not have a high-tech focus 
did not appear to have been reflected in practice. This was primarily 
true because almost all fields have some high-tech elements. If a 
business project was training students to use word processing it was 
teaching students to use "high tech" equipment. The same was true for 
an auto mechanics project that was teaching students to use 
sophisticated diagnostic equipment. There was almost no field in which 
certain high-tech elements did not exist. Because the Cooperative 
Demonstration Program made no distinction between operating equipment 
and understanding how it functions, the requirement that the project be 
high-tech in focus excluded virtually no training effort. 

4. Operation and Integration of the Demon stration within the 
Institution 

There were numerous reasons to examine whether projects were able 
to start and operate effectively, and whether they were eventually 
integrated into their institutions. Because this was a demonstration 
program, the regulations envisioned wide replication of project 
efforts. For a project to be replicable, however, it must be suffi- 
ciently active (or its products available) for others to adopt it after 
Federal funding ends. In addition, because the grant perio;, was short 
there had been concern that 18 months might not be sufficient for the 
projects to start, complete their activities, sM generate continued 
support. In Year One, several of the projects experienced start-up 
problems that delayed services and made completion of objectives 
difficult. As a result, the study team asked a number of questions 
designed to examine start up, project operation and continuation beyond 
Federal funding. 

There appeared to have been fewer start-up problems among the FY 
1989 grantees than among FY 1988 grantees, but problems remained. 
Problems were generally of three types-those that were primarily 
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I administrative, those that were the result of incomplete or unworkable 

' plans for the projects, and those that resulted from the economic 

I downturn that affected some communities and sectors of the economy* 

I Administrative Issues . According to survey results, most of the 

FY 1989 grantees said their projects would not have existed without 
I Federal support* Of the 27 projects, only four Indicated that the 

project would have existed without the Federal funds* Three of those 
I four were providing grant-related services prior to the start of the 

grant* At the completion of the grant period, 14 projects expected 
I that they would continue in their entirety, with another eight 

■ indicating that they would continue in a scaled-down form. Two 

I projects indicated that their Federal funding had not yet lapsed* 

I Projects generally began to provide direct services to clients 

within a few months after the start of their grants, but a few did not 
I provide services for many months* Excluding the three projects that 

were providing service to clients before the grant began, 12 projects 
I began providing services within three months of grant award, and nine 

more began within nine months* Three projects did not start, however, 
I until ten to 12 months after the grant awards (or six to eight months 

' before the grants were originally due to end!). 

g Nonetheless, the grant period appears to have been sufficient for 

I many of the projects to become institutionalized or find other sources 

of support* Twenty-two projects indicated they continued beyond the 
I end of the grant* The most common sources of support among continuing 

projects were funds from the grantee institution (11 respondents), with 
I private employers the second most common (eight r;^ondents had 

commitments from employers and four others were waiting for final 

■ agreements). Five respondents planned to charge students tuition for 
" project services. Of course, students were also likely to pay regular 
g college tuition in projects indicating they would continue with 

I institutional support. 

Interestingly, the ability of projects to provide services soon 
I after the grant award appeared to make little or no difference in 

whether they continued services beyond the grant period (see Table III- 
I 7). In fact, there was a slightly greater tendency for projects that 
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began providing services last to continue, with all of the latest- 
starting projects continuing in their entirety after the completion of 
the grant. Two of the three projects that indicated they were 
providing services before the grant started either received extensions 
to December 1991 or were no longer providing services at the time of 
the survey. 

Yet, when asked directly about factors affecting their ability to 
complete their original plans, the short grant period was cited as 
problematic by the largest number of respondents. Nine projects 
indicated that they encountered some problems in completing their 
original plans. The most commonly cited reason for implementation 
problems was that the 18-month grant period was insufficient, a reason 
cited by five of these nine projects. Also noted by more than one 
project were difficulties in staff recruitment and/or retention (two 
projects) and planned activities that proved inappropriate (two 
projects) . 

At the same time, however, ten projects indicated that they had 
accomplished activities they had not originally planned. The most 
commonly cited additional activity was curriculum development (five 
projects), followed by partner recruitment (four projects), and 
dissemination (three projects). Over half of these projects (six) 
indicated that they obtained additional funds that enabled them to 
undertake these unplanned activities. 

During site visits, the details of several administrative problems 
were apparent. At least three projects encountered some difficulty 
finding project directors or other staff. The reason was usually the 
same-it was hard to find talented persons willing to take on 18-mcnth, 
closed-end positions. In some cases the situation was complicated 
further because project positions were temporary or part time and, 
hence, did not include benefits. Overall, however, these problems may 
have occurred less often in the second year because the recession may 
have made people more willing to take less attractive positions (and 
also because several projects involved a continuation of activities 
begun under previous Cooperative Demonstration grants). 
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Problems in Implementing Project Design . Almost all the projects 
encountered some problems in implementing their original design, but 
the intensity of problems varied considerably. One of the most common 
problems among the projects visited was an inability to recruit special 
populations. Many of the projects thav had originally planned to iccus 
training oh disadvantaged persons either shifted their emphasis to 
other clienteles or simply recruited fewer minorities or nontraditional 
students than originally anticipated. The result, overall, was that 
the projects did not attract substantial numbers of minorities or 
nontraditional students by gender. 

The reasons for problems in recruiting varied. One project that 
originally planned to focus on JTPA-referred students found that its 
advisory committee of local businessmen insisted on a level of training 
that demanded academic skills few JTPA-referred students possessed. As 
a result, this project shifted its focus to training current employees 
of companies in the area. Ironically, it found that many of these 
students also lacked sufficient math skills to do the work, and 
instructors ended up teaching basic skills to the employee population^ 

Several projects made efforts to recruit black and Hispanic 
students but met with little success. One project attributed its lack 
of success to its inability to provide a technician credential to 
program completers, making it an unattractive 18-month investment. 
Another project was housed in a facility many miles from the grantee's 
main campus, in an area that had no public transportation. A project 
that depended on obtaining training from schools that were accustomed 
to payment from JTPA found that the schools were unwilling to take 
risky students-i)eople who might not finish the training-because they 
were used to the JTPA payment-for-performance approach. 

The results of these problems can be seen in the survey findings 
on project participants. Hales were the largest number of training 
recipients (see Table III-8). For the 79 percent of participants for 
whom gender data was provided, 67 percent were male and 33 percent were 
female. Further, males were concentrated in the projects providing 
training in manufacturing, while females were concentrated in the 
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projects providing training in business services and health care. Five 
of the six projects providing training in health care and three of the 
five reporting training in business services reported a female 
population of 50 percent or greater, compared with only two of the 16 
projects offering training in manufacturing and one of the four 
projects offering training in transportation (see Table III-9). 
Although most projects (19) indicated that they made special efforts to 
attract women, it did not appear that the projects provided a testing 
ground for nontraditional training by sex. 

For the 21 projects that reported the race of most or all 
participants, the majority of trainees were white. In these projects, 
whites constituted approximately 83 percent of participants, blacks 
were 11 percent, Hispanics were five percent, Asians were one percent, 
and American Indians/Alaskan Natives were less than one percent. 
Overall, then, whites constituted a greater percentage of participants 
in these projects than they were in the population. Fewer than two 
percent of participants were of limited English proficiency. Although 
19 grantees said that they made special efforts to recruit minorities, 
among the 21 projects that supplied information, the projects appear to 
have done little to demonstrate new opportunities for minorities. 
However, the seven projects with missing data on race include about 
half the participants. Even if minorities were well represented among 
their participants, they would not have received much training because 
these projects also reported the fewest training hours per person. 

The majority of trainees were employed full time during training. 
Of the 87 percent of participants for whom employment status was 
reported, 85 percent were employed full time during the period of 
training. Further, most of these persons were employed by the private 
sector partners that cooperated in the projects. Of the 80 percent of 
full-time employees for whom the employer was known, 92 percent were 
employed by a partner company. The total percentage employed by 
partners /iS probably somewhat less than this figure, of course, 
because employment by the partner was more likely to be known to the 
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grantee* Nonetheless* it Is possible to conclude that the main 
training recipients were full-time employees of partner organizations* 

Projects reported some problems due to the logistics of getting 
and maintaining plant and materials* For some projects, projected 
facilities and equipment contributions did not materialize. One 
project failed to gain the funds to build a clean room that was 
critical to Its training plan* Another did not obtain the large 
training facility It anticipated In Its grant application* One project 
found that loaned equipment was expensive to transport and maintain, 
presenting an unanticipated cost to the-project* 

Other projects simply could not execute the designs they 
originally proposed and had to make changes or curtail their offerings. 
One project planned to develop a specialized home automation system, 
but the developers of the system did not cooperate In curriculum 
development. As a result, the project staff had to alter Its plan and 
develop a generic home automation 1nst?ller curriculum Instead. 
Another project found that It was much harder than anticipated to 
"sell" businesses on the need for company-wide CIM reforms, and Instead 
offered short-term courses In specific computer-assisted applications. 
The three projects that planned to develop interactive videodiscs 
underestimated considerably the amount of time necessary for product 
development* None of the three had developed comparable software in 
the past* The project for high school students found it could not 
train students as groups during their internships, and asked students 
to maintain journals instead* The staff also found that internships 
planned to occur during the school year had to compete with many other 
out-of-school activities. Most of these problems were resolved, but 
for some projects the changes affected their ability to complete work 
within the 18-month grant time frame. 

The Economic Downturn * Finally, some projects faced problems 
because of the economic downturn, although problems appear confined to 
a small subset of the projects visited* Most commonly, anticipated 
partner organizations were unable to participate 'or participated at 
severely reduced levels* Further, partners were unable to provide jobs 
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for participants and completers. In one project, the projected partner 
laid off a third of its workers, including the person slated to be the 
main project liaison and an instructor in the program. In reveral 
projects, poor economic conditions led to low rates of student job 
placement. Nonetheless, the poor economy does not appear to have 
affected the majority of pro jects-especi ally those in industries such 
as health care where the conditions may not have been as adverse. 

These site visit findings were borne out by the survey results. 
About half the grantees (14) indicated that economic conditions in 
their communities had deteriorated since they applied for grants (mid- 
1990), but fewer indicated that the changes had affected implementation 
of the grant. When asked whether the decline had affected grant 
implementation, four said that the changes had resulted in fewer jobs 
for students and four indicated that it had resulted in problems for 
the institution. 

Grantees were slightly less likely to indicate that a decline in 
economic conditions in the specific industry or occupation in which 
training was provided had affected project operations. Over half the 
respondents (16) indicated a deteriorating economic condition in the 
specific field(s) of training, but only half of those indicating a 
negative economic change also indicated that it affected project 
operations. Six of these cited poorer job prospects for students as an 
outcome, while three' indicated a negative effect on the institution. 
Two grantees indicated that negative changes in the industry created a 
positive opportunity for the project by increasing training needs, and 
one grantee indicated that better economic conditions in the industry 
led to fewer training opportunities at the institution. These last 
responses may reflect a common finding that bad economic times, while 
creating poor job prospects, can sometimes translate into increased 
enrollments in training. Not only do the unemployed seek training, but 
persons who are concerned about losing their jobs may also enroll. 

Deteriorating economic conditions appear to have been most common 
among the projects that provided training in manufacturing (see Table 
III-IO). Seventy-one percent of the projects that provided training in 
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Table III-IO 

MAIN INDUSTRIES AND LIKELIHOOD OF ECONOMIC CHANGE OVER PAST TWO YEARS 

(NUMBER OF PROJECTS REPORTING) 



Industry-f 



Number of 
Projects* 



Construction 4 

Manufacturing 17 

Transportation 4 

Business Services 5 

Health Care 7 
Education 

I Multiple Industries 7 



Industry Conditions 
Improved Deteri orated 



Remained 
the Same 



0 
2 
0 
1 
1 
0 
1 



1 

12 
2 
2 
2 
2 
4 



+ Industries w/fewer than four responses omitted from analysis. 
* Adds to more than 27 because projects could select more than one 
industry. 



3 
3 
2 
2 
4 
2 
2 
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manufacturing reported that economic conditions in the industry of 
training had deteriorated since the grant application. Among those 
grantees providing training in health care, only 29 percent reported 
deterioration, while among those providing business services, 40 
percent indicated economic deterioration in the industry. As trainees 
in projects offering manufacturing training were the most likely to be 
full-time workers during the training, however, the economic 
deterioration may not have translated into immediate job effects. 

5. ExDortab ilitv of Proiect Activities and Products 

Because this was a demonstration program, the study team sought to 
determine what project features had applicability to, or could provide 
useful models for, other educational or employment training programs. 
What elements of these projects were worth replicating widely, as had 
been anticipated by the regulations governing this program? 

Survey Findings . The projects were quite divided in choosing 
features of their projects that could be used or applied by others. 
Respondents were provided with a list of possible project features and 
asked to rank up to three features of their projects they considered 
applicable or useful for others. The most commonly selected first 
choice feature was customized training for a particular employer or 
group of employers (six respondents). The second most commonly 
selected feature was a new or improved curriculum (selected by five 
respondents). "New or improved kind of training," "established or 
strengthened public-private partnerships" and "applied high technology 
equipment to the delivery of training" received four responses each. 
Three projects chose "expanded access to training for disadvantaged or 
underrepresented groups." Only one selected "model of school -to-work 
transition," which was a demonstration objective specifically 
identified in the Federal grants announcement. 

This wide range of responses can be interpreted several ways. 
Given that the program was intended to be a demonstration of public- 
private partnership for training in high-tech fields, the lack of 
consensus on exportable features is surprising. One might have 
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expected the partnership or training choices to account for a large 
number of first choices, but they account for only eight of 27 • The 
selection of customized training as a first choice is notable, because, 
by its nature, customized training is rarely applicable across 
industries or educational institutions. 

If all three choices were added together, however, establishing or 
strengthening public-^private partnerships appeared to be the feature 
most commonly selected as having wider applicability* Twenty projects 
considered partnership among their top three exportable features* 
Conversely, few projects that did not select customized training as a 
first choice selected it second or thirdnnaking it one of the lowest 
ranked choices overall* Beyond partnerships, however, no exportable 
feature was identified by more than half the projects, even with first 
through third choices added together* 

The lack of consensus on what is being '^demonstrated*' is 
troubling* While :t is not possible to reach conclusions from the 
responses to one question, it seems that grantees do not appear to 
share a common view of what they used Federal support to "try out*" 
And although all projects were expected to "try out" public-private 
cooperation for high-tech training, only 20 projects considered their 
new or improved public-private partnerships, and only 13 projects 
considered their training, as applicable to or providing a useful model 
for others* 

Site Visit Findings * The site visit teams tried to make an 
assessment of what could be replicated from the various projects. They 
considered two criteria* First, they looked for some evidence that the 
project succeeded in accomplishing its goals, i.e*, did it do something 
worth considering for replication elsewhere? If the project met the 
first criterion, it then needed a mechanism for facilitating 
replication-a product or other means for guiding others to implement 
the same set of activities* 

For some projects, particularly those concerned with curriculum or 
product development, evidence of project success and replicability 
could be easily determined* These projects were, essentially, "tests" 
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of the efficacy of the curricula or other materials/products (such as 
interactive videodiscv; the projects developed. The grants enabled the 
curricula or other products to be developed and installed in a pilot 
site or sites. The experiences of the users in those sites were 
observed and/or their comments solicited in a relatively systematic 
manner. Sometimes changes were made and the product was re-piloted. 
At some point, the developers were either satisfied or unsatisfied with 
the quality of the product. 

There were only a handful of projects that proceeded in this 
manner, however. They included the projects that developed curricula 
for home automation installers, semiconductor manufacturing employees, 
and metal fabrication. They also included the high school occupational 
clusters training and the two projects that developed interactive 
videodiscs that can be used in instructional settings. All of these 
products have been pilot tested through the projects in one or more 
sites and were nearing or at completion at the end of the grant period. 
Seeing how well they can be used by persons not receiving direct 
assistance from the developers could be the next step in determining 
their exportability. 

Beyond these projects, it is less clear what project activities or 
partnerships were exportable or applicable elsewhere. First, some of 
the closest partnerships-such as the two steel industry 
projects-provided a training experience and direct instruction that may 
not have wider applicability. At best, they present a general lesson 
for others-that colleges can perform valuable assistance in helping 
manufacturers convert to the use of high-tech equipment. Working 
closely with managers, instructors can design and deliver a mix of 
classroom instruction and OJT to employees. Unfortunately, most of the 
projects were not accompanied by the kinds of documentation or process 
evaluation that would explain how such partnerships should or should 
not proceed to be most effective. 

Second, beyond the curriculum or product development projects, the 
lack of systematic evaluations addressing project effectiveness makes 
it difficult to draw conclusions about project success (the 
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evaluation's first criterion for replication) ♦ Most projects did 
engage ••third-party" evaluators, but their evaluations either focused 
exclusively on process (l«e., they showed that projects met their 
obligations with respect to number of students, amounts of training, 
etc*) or lacked sufficient rigor with respect to assessing outcomes to 
reach conclusions* For example, a number of projects showed low rates 
of student p1acer,ent* Because the projects did not conduct evaluations 
with comparison or control groups, it was impossible to know the extent 
to which the lack of placements was due to the poor economy or to 
inadequate project activities or procedures. It should also be noted 
that several projects did not have completed ••third party^^ evaluations 
at the time of the site visiit (although the majority of projects were 
either completed or in their last month)* Despite the **demonstration** 
intent of the Cooperative Demonstration Program, evaluation was treated 
in a pro-forma manner by many projects; it was largely an afterthought 
to service delivery* 

A few projects did develop guides for implementing their 
procedures* For example, the project that assessed students and 
referred them for training to local schools and colleges produced a 
manual on its experience. Several projects provided the site visit 
teams with examples of curricula developed by staff supported, in part, 
with project funds. In the absence of any evaluations of the impact or 
effectiveness of these projects, however, it is hard to know whether to 
encourage others to use the same approaches, or even whether it would 
be worth the effort to test these interventions further. 
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D. Key, Findings 

1. Project Training 

Overall, projects focused on providing relatively short-term 
training. The typical participant received fewer than 100 hours of 
training* The projects included in the site visits can be grouped into 
a few basic categories: 



■ Short-term, skill-specific instruction for 
current employees of companies, or for 
persons already familiar with the field of 
training* This approach was most common 
among manufacturing projects, but it could 
also be seen in the refresher nursing course 
and in business projects teaching specific 
software applications, as well as home 
automation installation* Sometimes 
instruction was customized, tailored to the 
needs of specific employers* 

■ Medium-term, entry-level technician training. 
These were programs in which training ranged 
from a few months to a year, often aimed at 
helping unemployed or otherwise disadvantaged 
persons find jobs in a field* These programs 
cut across the industries but were quite 
common in business* The three projects run 
by JTPA-supported agencies fell within this 
category, one of which provided training in 
several fields* Some provided certificates. 

■ Longer-term programs aimed at two-year 
certification or degrees (associate degree, 
license). Some of these projects were aimed 
at institutional capacity-building and 
regular students, e.g., creating new or 
revising existing offerings within depart- 
ments of industrial technology or business/ 
computers in community colleges. One was an 
apprenticeship training project, and one an 
occupational program for high school 
students. 
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2. Partnerships 

Among projects included in site visits, intensities of partner- 
ships varied from two projects where single employers played critical 
roles in service development and delivery, to several projects in which 
employers did little more than attend meetings of an advisory 
committee* According to survey resu.vS, most partnerships fit one of 
two models: 

■ The partner was a customer of the project, 
e.g. the project provided customized training 
to the partners' employees; or 

■ The partner shared actively in the delivery 
of instruction and services. 

Projects with multiple partners were considerably more likely to report 
that partners were customers, while projects with few partners used 
partners in more ways, including. active sharing of instruction. 

3. High Technology 

Initial concerns that projects might not have a high-tech focus do 
not appear to have been reflected in practice. This is trje primarily 
because almost all fields have some high-tech elements. If learning to 
operate high-tech equipment is included as a training option, requiring 
that the project be high-tech in focus excludes almost no training 
effort ♦ 

4. Operation and Integration of Projects within Institutions 

FY 1989 projects reported fewer start up problems than did FY 1988 
projects. The 18-month time frame continued to present problems in 
completing project activities, however. Whether a project started 
providing services shortly after the award made little difference in 
whether it continued after the grant ended. The site visit teams 
identified several problems that cut across projects. They included: 
an inability to recruit special populations as planned, problems in 
getting and maintaining plant and materials, poorly developed designs 
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that could not be executed, and underestimating the time necessary for 
product development* Aside from the recruitment difficulties, most 
problems were resolved but had an impact on those projects' ability to 
complete work within the 18-month grant time frame. The poor economy 
did not affect the majority of pro jects-esp^iCi ally projects in 
industries such as health care where the conditions may not have been 
as bad. 

5. ExDortabilitv of Project Activities and Products. 

Survey results suggested that projects lacked consensus on what it 
was that the Cooperative Demonstration Program was ** demonstrating.** 
Only 20 projects considered their publicrpartnerships, and only 13 
projects considered their training, applicable to, or providing a 
useful model, for others. Site visits revealed that most projects 
simply did not conduct the kinds of evaluations that would allow 
possible replicators to determine whether the project was successful 
for participants. Projects that developed and tested discrete 
curricula or products were more likely to have some evidence of 
effectiveness and a product to disseminate. 
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ProblOHS \ 


Projact originally 1 
anviaionad cadres of 1, 
inatructora advlaing on V 
cospanyHide introduction of 
CIM. ThIa approach failed 
to develop. Conpanlea 
preferred Inatructlon in H 
apecific appllcationa -CAO, H 
CAN, and small training D 
aessions. 1 


According to project ataff, 1 
the adviaory board insisted 1 
on levels of training beyond H 
what could be handled by 
most disadvantaged U 
populationa, making H 
recruitment of these groups 1 
difficult. Promised H 
donatfona of warehouae space H 
and computer hardware for | 
the ATC were never made. u 
The original nine-week, 20- D 
hour-a-week format proved 
too condenaed for aeveral of 
the claases, ao they were 
reformatted to fit the 
atandard 18-week, three- ■ 
hour-a-week college aemeater 1 
format. 1 


Problema Included a major 
economic downturn In the 
community leading to poor 
ratea of Job placement. In 
addition, ataff indicated 
other problems including not 
screening training 
appllcanta for prior akill H 
levela or motivation. 1 
Minority recruitment waa 
difficult becauae partner 
achoola were accuatomed to 
JTPA payment, ao did not 
want to take atudenta 
unlikely to finlah training 
or different from other 
atuoenis. 9\ar% ot xreining 
at one inatitutlon was H 
delayed three montha while H 
it aearched for a new | 
inatructor (who then had H 
little time to aaaewble hfs H 
course). U 


OUtcmfnatlon 1 


Presentations were made at 
two national vocational 
education meetlnga and at 
CIM Alliance meeting. 
Videos produced. 


The project director gave 
two preaentationa on the 
ATC, one to State Vocational 
Education Directora, and the 
other to a conference of the 
California Induatrlal 
Technical Educational 
Aasoclatlon. He alao 
preaented information about 
the project plan to the 
Nation Coalition of Advanced 
Technology Centera in May 
1990. 


Project used newsletter to 
600 Indivlduala and 
organizations to publicize 
content, accompli ahmenta. 
There were few requeata for 
Information, however. Two 
other granteea viewed the 
project. Copies of CNC 
curriculum mailed to anyone 
requeating them. Staff made 
preaentationa at State and 
national meetlnga, e.g. AVA. 


Staff 1 


Project used existing 
instructors at the 
institution (grant paid for 
aubstitutes to free up 
reguliar ataff time to work 
on project). 


Fresno Connunity College had 
aeven ataff mcmbera working 
on the grant. One waa the 
project director, one was an 
admi nl at ra 1 1 ve adv 1 aor , one 
waa the project coordinator, 
and the other four were 
Instructors. 


Staff for the project 
Included a Director, 
recruiter/evaluator, and an 
equity development 
apecialiat. There were five 
inatructora from the partner 
schools, all but one of whom 
had worked on the previous 
Coop Demo grant. One 
project admlniatrator alao 
aerved as an Instructor. 


• 
• 

£ 


Partlclpanta were local bualnesa 
people and employees of those 
companlea. They were primarily 
male, white, and full-time 
employees. Fox Valley atudenta 
were alao participants In modified 
CIM-orlented curricula. 


The proposal atated that the 
project would aerve mainly 
diaadvantaged, unemployed, and 
refugee population; in fact, the 
project primarily aerved the 
employeea of partner companlea. 
Of the 124 atudenta who received 
training, 108 were men and 16 were 
women, 29 percent were Hiapanic, 5 
percent black and 8 percent were 
Aaian; 23 were unemployed. 


Project targeted special and 
nontraditional populations. A 
total of 204 atudenta were 
enrolled -including 125 high achool 
atudenta -end 171 completed 
(Including 115 high school 
students). Students were 19 
percent Hispanic, 16 percent 
Black; 3 percent Indian. Twenty- 
one percent were female. A total 
of 20 (non high school) 
psrticlpants were placed in 
jobs -and the 15 CNC machining 
atudenta were already employed. 




Fox Valley 

Technical 

College 


Freano (State 
Center 
Community 
College) 
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IV-1 

IV. PROJECT COSTS AND BENEFITS 

Were project costs reasonable relative to the projected or actual 
outcomes of the project? Costs were defined as the Cooperative 
Demonstration grant plus non-Federal cash match and in-kind 
contributions of the grantee. The outcomes of the project are defined 
as numbers of students trained (if training is the project focus) 
and/or the number of course hours developed (if curriculum development 
is the project focus). "Reasonableness" is judged by comparing costs 
and benefits among projects, nsi against some absolute standard. These 
analyses quantify major project outcomes and, where possible, the total 
costs of achieving them. No attempt was made to assign a monetary 
value to the benefits resulting from the project outcomes (e.g., the 
dollar value of learning a new skill). Also, the analyses excluded 
consideration of the costs in time and effort to students participating 
in the projects. 

This section is divided into four subsections. Subsection A 
identifies major cost/benefit issues and the operational definition of 
those issues. Subsection B defines and enumerates project costs, and 
subsection C defines and enumerates major project benefits. Subsection 
D compares costs with benefits in accordance with the major issues 
raised in Subsection A. 
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A, Major Issues 



Project costs and outcomes are aggregated to estimate overall 
costs and benefits. These are used to compute four measures: 

■ treatment costs ; total project costs minus 
planning and development costs; 

■ Project intensity : total treatment hours 
divided by the number of successful comple- 
tions; 

■ average unit cost of delivered services : 
total treatment costs divided by total units 
of service; and 

■ service cost per unit of outcome : total 
treatment costs divided by the number of 
successful completions. 

Operational definitions of treatment hours, successful completions, and 
units of service produced vary according to the type of treatment 
(e.g., training versus curriculum development). For projects focusing 
on student training, the number of treatment hours are the number of 
classroom contact hours; the successful completions are those students 
finishing the course; and units of service are hours of training per 
student. 

The four measures were analyzed only for the 19 FY 1989 projects 
for which data were collected during the site visits {see Section III). 
The site-visit teams collected limited cost and outcome data {e.g., 
project budgets and numbers of students trained) from each project 
through staff and partner interviews and review of budget and 
expenditure reports, project progress reports, and the mail survey of 
all FY 1989 projects. Project records showed direct expenditures from 
grant and from non-grant sources. Partner organization members were 
interviewed to determine direct expenditures or in-kind contributions 
to the project. Finally, project staff were interviewed to identify 
other in-kind contributions to the project, such as donated equipment 
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or release time from other organizations* Although not all in-kind 
contributions could be translated into dollars, the study team made 
rough estimates of the value of the contribution stated in the grant 
application or during staff interviews. 
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L Project Costs 



Project costs are the financial and nonflnanclal resources used 
for project activities. Since Cooperative Demonstration grants 
required grantees to contribute at least 25 percent of total project 
cost, project costs Include both the Cooperative Demonstration grant 
amount and the grantee match. Project costs often Included more than 
this, however. Some projects used existing Instructional services paid 
by the grantee as part of their regular operations. Others used staff 
and materials funded from other sources to supplement the activities of 
the Cooperative Demonstration project. These outside sources also 
should be Included In the calculations of total project costs. Thus, 

Total Project Costs « Federal grant + non- Federal match + 
outside project resources. 

Unfortunately, few outside sources could be Included In the calculation 
because the grantees had not tracked them. Thus, the analysis Is based 
only on the direct services and materials paid for with grant funds. 

1, Cost Categories 

The primary components of project costs used In the analyses were 
those In line-Item budgets. Including: 

■ Salaries and Wages; 

■ Fringe Benefits; 

■ Travel ; 

■ Equipment; 

■ Supplies; 

■ Contractual Services; 

■ Other Direct Costs; 

■ Total Direct Costs; and 

■ Indirect Costs. 

Salaries and Wages . Primary staffing costs were salary and wages 
paid to staff. Including annual salary and hourly wages for all 
employees of the grantee-or the partner organlzatlon-who work on the 
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project. Staff included teachers, instructors, administrators, other 
certified personnel, clerical staff, and support staff. 

Fringe Benefits . Employees usually received fringe benefits as 
part of their compensation package, including sick leave, annual leave, 
holidays, health insurance, etc. Fringe benefits are usually 
established by the institution as a percentage of total salaries and 
wages. 

Travel . Any triatvel costs were listed as a separate line item, 
including airfare, car rental, ground transportation, hotels, meals, 
and tips. 

Equipment . Although the program discouraged FY 1988 grantees from 
using grants to purchase equipment and prohibited it in FY 1989, some 
projects did purchase new equipment. Other projects used non-Federal 
sources to pay for equipment. 

Supplies . Projects required routine office supplies, specialized 
materials, and instructional materials. 

Contractual Services . Some projects hired outside experts or 
temporary personnel, including outside evaluators or specialized 
services provided by other companies through contracts. Rules 
governing the use of consultants (individuals) differ from those 
governing the use of companies (subcontracts), but both involve the 
external acquisition of services and are grouped in one category. 

Other Direct Costs . Direct costs that do not fit into the above 
categories are listed as "other costs" and may include space rental, 
telephone, and postage if they are not paid for indirectly (see 
Indirect Costs below). 

Total Direct Costs . The sum of all the direct cost categories 
yields the total direct costs of a project. 

Indirect Costs . Indirect costs are charges made by grantee insti- 
tutions for overhead items such as office space, heat, electricity, 
postage, accounting services, and administrative services. Grantees 
usually provided these to all projects and programs. To pay for items 
that are difficult to itemize, the grantee institution charged projects 
an indirect cost, or overhead, rate. The indirect cost was usually 
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based on a percentage of the total salaries and fringe benefits, but 
may also include other direct costs in the base. Under the FY 1988 and 
FY 1989 grant regulations, grantees were allowed to charge a maximum 
indirect rate of 8 percent. 

Total Project Costs , Total project costs were the sum of all 
direct costs and all indirect costs associated with the project. 

2. Source s of Funding 

Projects funded activities from three major sources: grant funds 
provided by the Federal government through the Cooperative Demonstra- 
tion Program grant award; non-Federal cash or in-kind resources from 
public or private organizations (e.g., the grantee or partner organiza-- 
tion) to meet the required 25 percent match; and funds or in-kind 
contributions not identified in the grant application nor reported in 
the project expenditure reports (usually underlying instructional or 
support services provided by the grantee as part of the regular 
educational program). In-kind resources included grantee staff time, 
partner staff time, equipment, facilities, services, materials, and 
information. Grantees did not always identify these resources because 
(1) the grantee already had satisfied the 25 percent match; (2) it 
didn't have the resources to track them; (3) [it needed them to offset 
any of the 25 percent matching funds that may be disallowed in a 
subsequent Federal audit]; or (4) the accounting system could not 
handle them. 

The total costs of each project and sources of funding are shown 
in Table IV-1. Total project costs ranged from $261,274 at Nebraska 
Department of Labor to $1,356,966 at Clackamas. It should be noted 
that grantees may have overestimated the value of the partner's 
contribution or given vague figures for equipment. 
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Table IV- 1 
TOTAL PROJECT COSTS 





PROJECT 


Faderal GRANT 
FUNDS 


LOCAL CASH 
MATCH 


LOCAL IN-KIND 
MATCH 


OTHER PROJECT 
RESOURCES 


TOTAL PROJECT 
COSTS 




Bronx CoiMunity 
College 


473,549 


0 


460,000 


0 


933,549 




CI 

CoMunlty 
College 


366,305 


0 


990,661 


0 


1,356,966 




CORD 


417,880 


8,270 


110,662 


117,755 


654,567 


D Fox Valley Tech 


437.727 


0 


0 


326,900 


764,627 




HaMpden County 


359,309 


0 


124,800 


0 


484,109 




Horn Builders 
Institute 


408,318 


0 


30,000 


0 


438,318 




Howard Ccanunity 
College 


214,924 


77,997 


15,325 


0 


308,296 




IlUnoU Eastern 
College 


178,270 


0 


105,400 


196,571 


480,241 




Indian Hills 
Col lege 


239,089 


98,434 


48,736 


0 


386,259 




LTV Steel Co. 


252,821 


839,319 


0 


0 


1,092,140 




Luzerne 
CoMMunf ty 
College 


234,174 


0 


180,218 


0 


414,392 


Nebraska Labor 
Department 


171,874 


0 


89,400 


0 


261,274 


North ClackaMas 
Schools 


257,274 


23,850 


138,930 


185,000 


605,054 


PAVE 






180,750 


0 


569,577 


Southuestern 
College District 


278,379 


238,213 


138,375 


0 


654,967 


State Center 
College District 


399,000 


0 


275,945 


0 


674,945 


UaUxxisee 
Coswunlty 
College 


243,328 


102,201 


0 


0 


345,529 


West Virginia 
Department of 
Education 


377,086 


11,884 


196,686 


0 


585,656 


West Virginia 
H Northern College 


471,808 


0 


161,439 


200,986 


834,233 



* total project costs » Federal grant ♦ local cash match ♦ local in-kind match 
♦ other project resources. 
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£^ Project Benefits 



Project benefits are improvements in the ability and/or employment 
status of students or In the quality of vocational education. Some 
outcomes are quantifiable (e*g*, the number of students successfully 
completing training), while others are nonquantifiable (e.g., a new way 
to identify skills needed by local employees). Nonquantifiable 
outcomes cannot easily be compared against project costs. 
Consequently, the following cost/benefit analyses use only quantifiable 
outcomes. 

Quantifiable outcomes vary with the type of activity and the focus 
of the project. For example, the benefits of curriculum development 
are new teaching modules or materials. Benefits of dissemination 
projects include "how- to-do-it" descriptions of the project, 
publications and presentations, or greater awareness by other 
vocational educational institutions. Similarly, benefits of training 
include the number of students with documented improvement in skill 
levels. 

The Statement of Work (SOW) for this contract identified three 
quantifiable outcomes to be measured: (1) the total number of service 
hours, (2) the number of successful participants, and (3) the number of 
person hours of services received per participant. For projects aimed 
at improving vocational education, primary outcome measures were: (1) 
the total hours of student training provided, (2) the number of 
students completing the training, and (3) hours of training received by 
each student. 

The grants funded other activities that provided services as well. 
Their value was measured through the three general measures defined 
above. The following are the quantifiable outcomes, by type of 
activity, that were measured for each project: 
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Activity 



Outcome 



Student assessment 



number of students assessed 



Training students 



number of students completing 
training 



Staff development 



number of staff trained 



Curriculum development 



number of course hours developed 



Skills identification 



number of industry skills 
identified 



Dissemination/diffusion 



number of other end users 
adopting product 



Partnerships/networking 



dollar value of partner(s) 
contributions 



The major outcomes from each type of activity are summarized in Table 
IV-2. These measures do not capture lU possible outcomes. Many 
important benefits could not be measured until after the end of the 18- 
month grant period. For example, while the number of students hired is 
one measure of the quality of training, no data on employment were 
available if the students had to undergo further training for a job or 
if they were already employed. The demonstrated mastery of technical 
skills is the ultimate measure of the effectiveness (along with job 
placement) of a new curriculum, but most projects did not conduct 
pretests because no .uch tests existed. Thus, available outcome 
measures often told what was done, but not how well it was done. 
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PARTNERSHIP 
DEVELOPMENT 


0 2 hospitals 1 
0 union 
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0 college | 
0 university 
0 Micron 
0 Sematech 


0 IBM 


0 10 partners 1 
incl. com. 1 
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voc. h.s. 1 
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0 2 hospitals 
0 college 


0 Intel 
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DISSEMIN- 
ATION 


0 replication 
manual 


0 conference 
present « 

0 ERIC 
article 


0 conference 
present. 

0 curriculum 
to be sold 


0 5 videos 
0 conference 
present 


0 proj, news- 
letter 
0 visitors 
0 conferences 


0 in progress 


0 videodisc 
to be sold 

0 conference 
present. 


0 by request 
0 conference 
present. 


0 conference 
presentation 
0 copies to 
ERIC 


SKIUS 
IDENTIFIC. 


0 continual 


o 


0 advisory 
comnittee 

0 CORD 
research 


0 customized 


0 done by 
proj. staff 


0 partner 
0 pi lot test 


0 advisory 
coftmittee 


0 experts 
consulted 


0 partner 
feedback 

0 advisory 
board 


ll 

AC > 


0 18 courses 


0 4 courses 
developed 


0 23 courses 


0 7 courses 


0 one course 
developed 


0 one course 
developed 


0 Interactive 
Videodisc 
developed 


23 

E 

O 


0 modified 
existing 
computer 
and math 
courses 


>- 

ac 
tu 

1 

U. «J 

Ui 


o 


0 5 trained 
0 215 hours 


0 none 


0 conferences 
0 workshops 
0 site visits 


0 one teacher 
trained OJT 


0 teachers 
trained as 
part of 
training 


0 none 


0 7 staff 
0 140 hours 


0 none 


STUDENT 
TRAINING 


0 11 students 

trained 
0 31,680 hours 


0 83 trained 
0 90,000 hours 


0 40 students 

trained 
.0 7564 hours 


0 216 students 

trained* 
0 322,310 

hours 


0 107 students 
0 151,298 
hours 


0 35 students 

trained 
0 1,890 hours 


0 29 students 

trained 
0 145 hours 


0 12 students 

trained 
0 5,966 hours 


0 70 students 
0 25,470 hours 
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ASSESSMENT 


0 national 
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0 prior educ. 


0 placement 

exam 
0 interview 


0 pre and 
post asset 
tests 
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rcqs. 
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0 evaluate at 
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0 pre, mid, 
and post 
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D, Cost/Benefit Analyses 



Ideally, the project team would determine cost effectiveness by 
matching costs to specific activities and outcomes. However, the team 
could not collect cost data for each outcome separately for two 
reasons. First, grantees did not code expenditures by activity because 
their accounting system did not operate at that level of detail, i.e., 
expenditures were aggregated only according to budget line items. 
Second, several activities usually contributed to the same 
outcome-projects implemented activities or one activity was used to 
support more than one objective. 

The team analyzed the three activities for which cost data were 
available through the grantee's accounting system or the final contract 
budget: planning and administration, student training, and curriculum 
development. Planning and administration costs include wages of the 
project director and clerical staff, associated fringe benefits, other 
direct costs associated with administration, and indirect costs. 
Student training costs include the wages of Instructors and other 
specialists, associated fringe benefits, other direct costs (e.g., 
textbooks, supplies, travel, and stipends), and indirect costs. 
Curriculum development costs include the wages of instructors and 
curriculum development specialists, associated fringe benefits, other 
direct costs (e.g., training workshops, travel, and printing), and 
Indirect costs. 

1. Treatment Costs 

The first major analysis separated planning costs from the costs 
of providing the service. Treatment costs equal total costs less 
planning and administrative costs: 

Treatment Costs « (total project costs) - (project director 

wages + clerical wages + fringe benefits + 
other direct costs + related indirect costs) 
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Treatment costs and planning/administration costs for each project are 
displayed in Table IV-3. The proportion of total project resources 
devoted to planning and administration ranged from 9.5 percent at LTV 
Steel to 48.0 percent at PAVE. 

2. Project Intensity 

The intensity of the project (i.e., the number of successful 
outcomes relative to the effort expended to accomplish those outcomes) 
is defined as: 

Project Intensity = total treatment hours 

number of successful completions 

The effort to train students was the sum of the hours students spent in 
training in all courses. The number of successful outcomes was the sum 
of the students completing training. In projects where the treatment 
was staff training, the number of successful outcomes was the number of 
teachers completing training. Project intensity for each activity for 
which data were available-training students and training staff-are 
displayed for each project in Table IV-4. The amount of training per 
student completed, project intensity, ranged from 16.92 hours at 
Southwestern Community College to 7,000 hours at LTV Steel. Staff 
training, offered formally at only three sites, ranged from 20 hours 
per teacher to 53.5 hours per teacher. 

3. Average Unit Cost of Services 

The average cost per unit of service is the total cost of the 
service divided by the number of units provided: 

Average Unit Cost « total treatment cost 

of Service total units of service provided 

For example, total service costs for providing student training was the 
sum of the project costs for staff, materials, overhead, etc. for all 
the courses. Total service costs for curriculum development was 
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the sum of project costs for staff, testing, reproduction, overhead, 
etc*, for all the courses produced. Total units of service provided 
was the sum of all students entering training or the number of new 
courses. 

Average unit costs for student training and curriculum development 
for each project are displayed in Table IV-5. The average unit cost 
for student training ranged from $.45 at PAVE to $40.47 at State 
Center. 

Comparing the average cost-per-hour-of-training across projects 
may create an unfair comparison because of variations in the intensity 
of the training and the number of students being trained. These 
differences affect the comparison of costs for curriculum development. 
The average unit cost for curriculum development (cost per course hour) 
ranged from $1.69 at the West Virginia State Department of Education to 
$24,179.90 at Howard Community College. However, comparing the average 
cost-por-unit-hour for curriculum development also may be misleading. 
Howard's course was an interactive videodisc, which had a much higher 
initial development cost due to the technology used. 

To provide a more accurate comparison, the analysis should 
compute the average cost per hour of training per student trailed . 



Table IV-6 shows the per-hour per-student costs for the projects. 
The per-hour per-student costs range from $.001 at West Virginia 
Northern to $9.24 at Luzerne Community College. The costs for the 
remainder of the projects tended to concentrate under $1.00 an hour. 

4. Service Cost Per Unit of Outcome 

The service cost per unit of outcome is defined as total service 
costs divided by the number of completions: 

Service Cost « total service costs 

per Unit of Outcome number of successful completions. 



Average Unit Cost 
of Service per 
Unit of Outcome 



total units of service provided 



total service costs 



number of completers 
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The number of successful completions Is defined as the sum of the 
students completing the training or the number of courses successfully 
developed. The unit service costs for training students and curriculum 
development are shown for each project In Table IV-7. Service costs 
for student training ranged from $190 at West Virginia Northern to 
$62,971 at Bronx Community College. For curriculum development, the 
cost per course ranged from $233 at Indian Hills to $241,799 at 
Hampden. The relatively high cost at Hampden County again was due to 
the high cost of the Interactive videodisc. 

In summary, the answer to the question "are project costs 
reasonable In relation to project outcomes?" appears to be yes for all 
projects. The per-unit and per-outcome costs for all the other 
projects tended to cluster In the same area even though total costs and 
project Intensity varied substantially. 
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V, ISSUES IN FEDERAL PROGRAM MANAGEMENT 



Although the Cooperative Demonstration Program (High Technology) 
has ended, there are important lessons to be applied to future demon- 
strations sponsored by OVAE and the Department*^ The experiences from 
the 27 FY 1988 and FY 1989 grantees visited by the study team suggest 
at least two groups of issues in Federal management of cooperative 
demonstrations: 

A. Specification of the purpose of and roles in the 
demonstration 

■ Policy-implementing versus policy-formulating 
demonstrations; 

■ Use of evaluators; 

■ Partner commitments; 

■ OVAE support of grant changes; 

■ Utilization of all available resources; 

B. Administration of the application and grant award processes 

■ Instructions to applicants; 

■ Timing of the grant award; and 

■ Length of the grant; 

Each of these issues is discussed below. Based upon the results of 
this evaluation, the study team recommends alternatives for the design 
and administration of future demonstrations • 
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^ The Cooperative Demonstration Program is currently authorized under 
section 420 A of the Carl D» Perkins Vocational and Applied Technology Education 
Act of 1990 (PA* 101-392) and implemented by 34 CFR Part 426* 
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A. Clarifying the Demonstration Program 



The Purpose of the Cooperative Demonstrations 

Although titled "Cooperative Demonstration Programs," the 
authorizing legislation did not clearly define the term demonstration 
or the intent of the legislation. Instead, the legislation referred to 

programs and projects which support model 
programs. • .examples of successful cooperation 
between the private sector and public agencies in 
vocational education. . .programs to overcome 
national skill shortages... and activities such as 
institutional and on-the-job training, supportive 
services, and other such necessary assistance... 
(Section 411). 



The legislation did not specify whether projects were to demonstrate 
that an intervention could be successful if it hasn't been tried 
before, could be improved in its original site if already in operation, 
could be successful in a new site if already implemented elsewhere, or 
some combination of these intentions. However, Section 411(c)(2) did 
require that all funded programs be "capable of wide replication by 
service providers,** suggesting that Congress intended the programs to 
demonstrate approaches that had applicability beyond just one grantee. 

The program regulations also were unclear about what was to be 
demonstrated. The regulations specified that projects. 



...must be designed to demonstrate ways in which 
vocational education and the private sector of the 
economy can work together effectively to assist 
vocational education students to attain the 
advanced level of skills needed to make the 
transition from school to productive employment... 
[34 CFR 412.10(a)(2)(ii)] 



This broad definition allowed a wide variety of projects to be 
funded. Although all projects satisfied the general conditions of the 
regulations, it is questionable whether all projects met the intent of 
the regulations and the Perkins Act. For example, is a grant to a 
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single company to retrain current employees In training programs the 
company would have done anyway a legitimate purpose under the program? 
OVAE should dearly state whether the demonstration is to field test 
previously identified public/private partnerships or to develop new 
models of such cooperation. 

A key characteristic of demonstrations is that they involve •'•••an 
innovation operated at or near full-scale in a realistic environment" 
(Glennan, et aK, 1978). Further, demonstrations can be divided into 
two types: policy-implementing or policy- formulating* Policy-imple- 
menting demonstrations take research ideas and put them into practice 
in the real world, while policy-formulating demonstrations generate new 
ideas and approaches based on actual field experiences (Yin and Sabol, 
1991). The knowledge from policy-implementing demonstrations is 
focused on the experiences and outcomes from implementing ideas 
previously found effective under controlled research conditions. The 
knowledge from policy- formulating demonstrations is focused primarily 
on identifying ideas which can be further tested under research condi- 
tions. In both cases, demonstrations cannot be expected to establish 
cause-and-effect relationships because the real-life settings involve 
confounding events (Ginsburg, 1989). 

Other Federal agencies have been more specific in their stated 
intent for funded demonstrations. For example, the Office for 
Substance Abuse Prevention, U.S. Department of Health and Human 
Services, stated that the purpose of one demonstration program was to: 

1. Test the feasibility of implementing 
previously untested innovative prevention 
strategies that hold great promise for 
expanding our repertoire of strategies and 
interventions in the prevention of alcohol 
and other drug abuse among high-risk youth 
populations; or 

2. Assess the program effectiveness, repli- 
cability, and general i lability of 
knowledge-based, established strategies for 
the prevention of alcohol and other drug 
use among high risk youth populations. 
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including those derived from previous OSAP 
experience but not systematically 
evaluated. 



Applicants were required to: 



Indicate whether this proposal is to (1) test 
the feasibility of implementing innovative 
prevention strategies; or (2) assess the 
effectiveness, replicability, and gener- 
al liability of knowledge-based, establisheo 
strategies for the prevention of and/or early 
intervention against alcohol and other drug 
use among high-risk youth. If this applica- 
tion is proposing an innovative strategy, it 
is incumbent on the applicant to explain from 
the literature review what makes this ap- 
proach innovative. If this application is 
designed to assess effectiveness, gener- 
al izability, and replicabi -Hy of established 
strategies discuss what is k^own about the 
established strategies and how t! is applica- 
tion addresses general izability and replic- 
ability of those strategies. 



Partner Commitments 

Projects funded by the Cooperative Demonstration Program were to 
demonstrate how public agencies in vocational education and the private 
sector could effectively and successfully work together. The regula- 
tions do not suggest what role the private sector should play, nor is 
the program designed to produce information on what roles were most 
effective for the private sector. This meant that each project was 
left to deal with developing the private sector role as it saw fit. 
The private sector was to be involved in the planning as well as the 
operation of the project (five points were awarded in the application 
for private-sector involvement). The role of the private sector was 
not well defined in the program regulations, and, as discussed in 
Section III, varied widely across projects. Among the FY 1989 
projects, the private sector was: the grantee, the referral source of 
students, and employer of program graduates, a source of internships, a 
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member of the advisory group, a supplier of equipment, or a source of 
Instructors* 

The lack of specificity In the regulations and application 
requirements regarding private-sector Involvement appears to have been 
problematic for several projects. Because the regulations were vague 
about what was acceptable private-sector Involvement, applicants 
similarly were vague In their descriptions of how private companies 
would be Involved In their projects* The vagueness In the applications 
sometimes was an Indication of lack of planning on the part of the 
grantee* Other times It was the result of the grantee not being able 
to get the private partners to commit to hiring students or to 
activities other than advising the project* 

Grantees reported problems In Implementing their projects because 
of changes or problems In the partner's role* Some of these problems 
might have been avoided through additional planning In the application 
process* One problem was that the role of the partner was left very 
general during the application, and the partner was reluctant to be 
specific after the grant started* Some grantees overstated what the 
partner was to do* For others, a change of conditions made It 
Impossible fcr the partner to fulfill Its planned role. 

The partnerships described In the applications appeared to be much 
stronger and more formal than those actually observed during the site 
visit* The statement that the applicant '*has been working with Ajax 
Corp*** for several years can mean many things* It can mean that Ajax 
has supplied Instructors or equipment or It can mean that Ajax has 
hired graduates of the Institution* 

Partners were asked to submit letters of support for the applica- 
tion* However, the partner might never have seen what the applicant 
stated In the application and might not have been willing to provide 
the kind of support Indicated* On the other hand the partner may have 
made substantial oral promises that were not committed In writing* The 
applicant assumed that It had a formal commitment based on the oral 
statement, but the partner did not feel It was a firm commitment* 
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The applicant should be required to have a signed document from 
each partner stating that the partner has read the proposal and is 
willing to commit to the support stated therein. After the award, and 
during the planning stage, the grantee should enter a formal agreement 
with the partner detailing precise responsibil ities. Otherwise the 
project can be left without adequate support* 

Partner support has to be clarified. Does it mean a guarantee to 
hire? to refer students? to provide equipment? to provide technical 
assistance etc.? Many grantees considered as partners the businesses 
that simply hired its students, even though the businesses had little 
involvement in the training process. 

Use of Third-party Evaluators 

Grantees' use of third-party evaluators was limited to verifying 
project activities. Although all projects had third-party evaluators, 
they were seldom used to measure the effectiveness of project 
activities or help determine the relative worth of the components of 
the project. Instead, grantees used the outside evaluators to confirm 
that activities had taken place. Most projects set aside $3,000 to 
$7,000 to hire a single outside person to conduct the evaluation. The 
evaluator observed several classes, talked to several students and 
faculty > and produced a report of a few pages. In a few instances the 
evaluator was used to assess the curriculum materials adopted by the 
training rather than the project itself. 

The relationship between the evaluator and the project was not 
always unbiased. In one instance, the third-party evaluator was the 
same organization that had provided assistance in writing the grant 
application. Afterward, the evaluator provided technical assistance as 
well as evaluation services. The two roles had the potential of being 
confl icting. 

Utilizing All Available Resources 

Many of the projects engaged in curricula development either as a 
primary objective of the grant or in response to requests for specific 
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training from a partner organization* In almost every case, the 
project developed the new curriculum without drawing from existing 
curriculum materials from other institutions, from ERIC, or from the 
OVAE regional curriculum coordinators. When asked why the project did 
not use-or at least review-the curriculum materials available 
elsewhere, project directors responded either that their training was 
unique and that materials developed elsewhere would probably not apply 
or that there were no materials elsewhere. A few project directors 
reported that they checked with other grantees, but only one reported 
conducting a systematic search for materials. 

The failure to conduct a systematic search of other institutions 
and the lack of communication among the grantees resulted in some 
duplication of effort in curriculum development. Although some pro- 
jects had unique elements, the program requirements for demonstrating 
successful cooperation in vocational training and for repl icabil ity 
suggest that the project already should be grounded in the existing 
literature and curriculum for the subject in their training area. With 
the scarcity of Federal and non-Federal resources, it is important to 
minimize any duplication of effort. 

To reduce or eliminate duplication of effort in curriculum design, 
OVAE may want to require applicants to show evidence of a systematic 
search for curricula as part of their proposal development. If OVAE 
thinks such a systematic search is too burdensome during the applica- 
tion stage, then the search should be required as part of the initial 
six-month planning grant described earlier. Even if a search is 
conducted during application preparation, an updated search should be 
made during the planning grant stage to identify any new curriculum 
packages published during the previous 12 months. 

OVAE Support of Changes bv Grantees 

Several projects encountered operational problems during the grant 
period. Problems included a deteriorating or shifting labor market, 
withdrawal of a key partner, or discovery that an activity was more 
difficult or expensive than originally estimated (e.g., producing a 
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videodisc)* In reviewing the implementation history of the project, 
the study team thought the problems were probably severe enough to 
warrant a change in project design or objectives, although such changes 
were made in only one case. During the site visits, the team asked 
project directors and staff why they didn't request a change in the 
grant's scope to keep up with changing conditions* Almost all said 
that they thought they had to fulfill the terms of the. grant 
application and that any requests for changes would be considered a 
sign of failure or simply not approved ♦ This misperception on the part 
of project staff resulted in the project continuing on an 
inappropriate-and ultimately unsuccessful-path. 

There were several reasons why the OVAE program managers might 
have been unaware of all the implementation problems encountered by 
grantees. First, not all projects submitted the required quarterly 
progress reports, and those that did often did not include a complete 
discussion of any problems being encountered. Second, even during the 
periodic telephone calls by program managers to the project, project 
directors might have been reluctant to share the true extent of their 
problems because of a decision by the grantee to "put the best face 
forward." Third, the evidence to justify a change might not have been 
conclusive at the time of the problem, and project staff were 
optimistic that they could still correct the problem later in the grant 
period. Finally, the large number of grants (30 to 35) that had to be 
monitored by each Federal program manager plus the additional workload 
of reviewing new applications and drafting revisions to program 
regulations meant there was little time to do more than cursory study 
of the grantees' reported activities. 

OVAE may want to consider two solutions to the problem of grantees 
not divulging serious implementation problems. First, OVAE program 
managers should discuss potential implementation problems with grantees 
at their kick-off meeting in Washington, D.C. The discussion should 
include both the nature of the problems encountered by other demon- 
strations (not just the Cooperative Demonstration Program) and the 
conditions under which it is important and acceptable to request 
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changes in the scope of the grant. Second, program managers should use 
the monitoring telephone calls to discuss both the accomplishments of 
the project and the problems (potential or actual) preventing expected 
accomplishments. If project directors are open about the problems they 
encounter, then the program manager will be better able to suggest 
solutions or put the project director in touch with other grantees 
having similar problems. This more open, communication however, will 
require program managers to become more active in the internal 
operations of the project. 
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B. Administering the Application and Grant Award Processes 

Instructions to Applicants 

The initial instructions to applicants are important for cor?veying 
information about program priorities* The instructions also can be 
useful in ensuring that applicants give sufficient forethought to their 
projects and avoid problems during implementation. The OVAE program 
staff has a certain amount of discretion in terms of the amount of 
information it provides in the application and the manner in which it 
is pi8sented. 

Instructions to applicants can be more specific than those 
currently in effect. The study team examined similar regulations for 
demonstrations in both the U.S. Department of Labor (DOL) and the U.S. 
Department of Health and Human Services (DHHS). The instructions 
issued by the Office for Substance Abuse Prevention (OSAP) in DHHS 
offers an interesting mc'el . Overall, the OSAP instructions are much 
more comprehensive and give a better understanding of how an applicant 
should respond to the grant announcement. As. described earlier, the 
OSAP grant announcement provides a clear definition of what a logic 
model is and a sample outline for conceptualizing and developing a 
demonstration grant using the components of a logic model. 

One of the OSAP grants announcement's major components is 
evaluation. The Anti-Drug Abuse Act of 1988 stipulates that priority 
should be given to demonstrations that employ appropriate strategies 
for evaluating the effectiveness of their proposed project. OSAP will 
only support projects with a "well --developed" evaluation plan. All 
evaluation plans are expected to include both process and outcome 
evaluations: 

Process evaluation is a quantitative and qualitative 
description of the intervention, target population, and staff 
of a project from inception. Process evaluation should 
clearly and comprehensively document the relationship of the 
resources and program activities to the project objectives so 
as to permit mid-project adjustments as needed to optimize 
project implementation and ultimate replication. The 
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evaluation plan should include a description as to how the 
components of a process evaluation have been or will be 
obtained/collected and maintained. The following are 
components of a process evaluation: 



!• Problem : the process by which the problem 
was defined as to (a) what being demon- 
strated, (b) selection of risk factors to 
be addressed, (c) analysis of process by 
which population becomes at risk; 

2* Target Population : including demographic 
and other relevant characteristics, case 
findings and retention strategies; 

3. Goals and Objectives of the Project to be 
Evaluated : the process by which the goals 
and objectives of the project to be eval- 
uated were selected; 

4» Staffing Patterns : staff characteristics 
and qualifications including that of pro- 
ject director; supervision patterns; staff 
selection processes; staff activities and 
work schedule; 

5. Referral and Case Finding Patterns (if 

a pplicable) : number, type, characteristics 
of referrals to and from the project; 
participating agencies in the project 
including the development of inter- 
organizational linkages with these 
agencies; 

6» Intervention : frequency, duration, type of 
contract; client flow, intervention mate- 
rials, manuals, staff training; staff and 
client perceptions of the interventions and 
objectives of the project; 

7. Cost Data : funding sources, cost per ser- 
vice, cost per client; 

8» Evaluation Procedures : monitoring instru- 
ments, need and risk assessment instru- 
ments; feedback mechanisms to director, 
staff, community representatives; and 
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9. General izabilitv of Program Findings for 
Program Dissemination : Manuals and/or 
curricula that will be produced. 

Outcome Evaluation: The evaluation plan should be 
detailed and clearly articulated. It should present an 
evaluation design appropriate to the project and of 
sufficient rigor to permit drawing valid conclusions 
concerning the effectiveness of the various intervention 
strategies. Outcomes variables shou^'d be derived from the 
logic model . 



At least one grant demonstration program requires applicants to 
describe in detail the projected performance indicators for the 
project. The U.S. Department of Labor's "Disabled Program" requires 
advanced estimates of performance indicators similar to those used in 
the cost benefit portion of this evaluation: 



■ Placements . Indicate the number of trainees 
who will be trained and placed in unsubsi- 
dized employment upon completion of the ser- 
vices provided (which cannot be less than 
100); 

I Average Cost Per Entered Employment . In 
relation to the total Federal costs proposed, 
indicate the expected average cost for each 
trainee who will be trained and placed into 
unsubsidized employment; 

■ Average Hourly Wage . Indicate the expected 
hourly wage that will be received by trainees 
upon completion of the program; and 

I Projected Performance Indicators shall be 
provided on a quarterly basis and for each 
project site (Announcement SCA/DAA-92-001) . 



Timing of the Grant Award 

The timing of the notification of the grant award and the starting 
date for the award have important implications for project implemen- 
tation. Most. academic institutions plan their staffing and activities 
according to the academic-year calendar, i.e.,, the academic year starts 
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in the fall and ends in late spring or early summer. The institution 
may offer summer activities, but these are secondary to those conducted 
during the regular school year. Academic institutions plan in the 
spring for the following year and try to have both activity schedule 
and staff assignments settled by May or June preceding the September 
start. 

Projects that start at the beginning of the academic year seem to 
be easier to implement than those starting mid-year. The FY 1989 
grantees were notified of their awards in October 1989 and most 
negotiated award starting dates of January 1, 1990. (Two projects 
delayed their date until July 1990 because of existing Cooperative 
Demonstration grants that were still in progress.) The difficulty with 
a January start is that all available grantee staff are already 
assigned to other projects, are in the midst of those assignments, and 
are reluctant to shift to a new assignment. If the project goes 
outside the grantee to hire project staff, there are fewer qualified 
staff available. 

If new staff are hired, or even experienced staff reassigned, the 
planning for the project is taking place after other institutional 
events and programs have been established. Therefore, staff and other 
resources may not be available for use by the project as proposed. It 
might help the project to align the start of training activities with 
the start of the school year. 

The main exception to scheduling project start-up for the 
beginning of the academic year is curriculum development projects that 
require significant amounts of instructor time. In this type of 
project, the start of the curriculum development phase should coincide 
with summer vacations when schools are out and teaching loads are 
light. In the summer, instructors have more time, are better able to 
concentrate, and will be able to integrate their new curriculum with 
their lesson plans for the next year. 

Although grantees have two to three months advanced notice of the 
grant start by virtue of entering into negotiations with the Depart- 
ment, grantees often do not start planning or hiring until the grant 
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award officially starts. Several grantees commented that they knew of 
funding (Federal or State) that was canceled just before the award was 
made, and so the grantees no longer initiate work based on verbal 
agreements. Grantees wait until the grant has officially started 
before they began to look for staff, rent space, assemble equipment, or 
negotiate contractual agreements with partners. For example, if a 
grant award is made on September 30th for a project start date of 
December 1st, the grantee will wait until December 1st to begin any 
activities. 

Length of the Grant 

The time period for the grant also has important implications for 
the activities that can be carried out and the ability to draw conclu-- 
sions about outcomes from the grant. The time period for the Coopera- 
tive Demonstration grants was 18 months, with some projects requesting 
three- or six-month extensions. For grantees with no existing staff or 
training activities to assign to the project, between three and six 
months of the grant period had to be devoted to hiring staff, estab- 
lishin:; office and contracting procedures, and planning training. For 
grantees with existing staff or similar training activities already in 
place, startup costs were not as high but the training offered was 
often more extensive (part of at least a two year training program). 

Most grantees agreed that 18 months was not enough time to 
accomplish anything other than customized training. A more workable 
arrangement would dissaggregate the grant into two separate-but 
related-grants totaling 24 months. The first grant would be a six- 
month planning grant during which the grantee could update the labor 
market information on which the training is justified. 

There can be as many as 18 months from the time the labor market 
information is collected and the award starts. During that time, local 
market conditions can change dramatically. Business can shut down, new 
employers move into the area, competing programs start, or the 
technology of the field can change. The planning grant also would 
allow the grantee time to hire the staff necessary to run the project. 
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The advertising, interviewing, hiring, and orientation can take as long 
as four months. Once project staff have been hired, they will require 
an additional two months to establish routine office procedures. 
Finally, the planning grant allows time for the grantee to negotiate 
specific roles and responsibilities with the private partner. If the 
proposed private partner is unable to fulfill commitments made during 
the application process, the planning grant then gives the grantee time 
to find an alternative partner(s). 

second grant would fund actual training or other proposed 
activities for 18 months. Because grantees currently find it difficult 
to attract project staff if there is no certainty of long-term employ- 
ment, the operating grant would have to be virtually guaranteed unless 
their plans were not adequately developed. However, the certainty of 
24 months of funding must be balanced against the need for projects to 
demonstrate appropriate and efficient planning at the end of the 
planning grant. The OVAE program managers (and perhaps the original 
review panels) would be required to review and approve the operating 
plans before the second grant could be released. 
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VI. FINDINGS AND RECOMMENDATIONS 



This section summarizes the major findings from sections II, III, 
and IV, and answers the three evaluation questions framing the study. 



A. Did the Grant Applications Pr esent a Clear 
and Coherent De <fiq" for a Project? 



■ Of the 23 FY 1988 applications reviewed, 
three were judged high in the likelihood of 
being successful (being both logical and 
plausible), ten were judged as moderately 
likely, and ten were judged less likely. The 
quality of the awarded applications improved 
in the second year. Of the 30 FY 1989 
applications reviewed, nine were judged high 
in the likelihood of being successful (being 
both logical and plausible), 16 were judged 
as moderately likely, and five were judged 
less likely. 

I Key terms in the application package and 
program regulations were not operationally 
defined, resulting in wide disparities across 
projects. 

I Applications frequently did not state clearly 
what was to be demonstrated. 



Based on these findings and other information collected during the 
evaluation, the study team recommends that: 



1. Program regulations should be more 
precisely worded to convey a clear meaning 
of key terms such as "demonstration" and 
"high-technology"; 

2. Applicants should be instructed to clearly 
show the logic and plausibility of their 
project designs; 

3. If logic and plausibility are important to 
the Department, they should be assigned 
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points in the evaluation criteria (much 
like the points assigned for organization 
capacity); and 

4. The panel reviewing the applications should 
include experts familiar with the high 
technology field or activities being 
proposed. Such experts could identify 
activities that are not feasible within the 
time frame or resources proposed. A bank 
of experts can be identified and 
tentatively invited prior to the receipt of 
the applications, and those with direct 
expertise can be appointed to the panels 
once the technical areas have been 
identified. 
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B. Were the Prp . iect Qpsinns SuccPs.f.mY Tn,p i».,.ntrfi? 

■ FY 1988 grantees were generally surprised 
that they were funded and were not prepared 
for the immediate start of the project: FY 
1989 grantees were less surprised because of 
better monitoring of the award process (and 
several were FY 1988 grantees). 

■ Grantees who were already operating similar 
projects were able to implement their pro- 
jects faster and more successfully than pro- 
jects starting from scratch. The availabi- 
lity of existing partnerships, staff, and 
institutional support was a key factor in the 
project's success. 

■ Local labor market conditions significantly 

applications were 
submitted and made it difficult to implement 
the proposed internship, job placement, and 
recruiting components. It appears that 
applicants may not have fully disclosed or 
understood local economic conditions in their 
application. 

■ Grantees who had no prior working relation- 
ship with the proposed partner(s) had diffi- 
culty getting cooperation or support from 
that partner(s). The problems in obtaininq 
cooperation from the partner organization 
were exacerbated by local labor market 
conditions. 

■ Projects with larger numbers of private part- 
ners used those partners in less intensive 
ways, while projects with fewer partners used 
them more intensively. Projects with fewer, 
more involved partners tended to be more 
successful. 

■ Grantees targeting hard-to-serve populations 
tended to shift their focus to less chal- 
lenging students once they realized the dif- 
ficulty of what had been proposed. A few 
projects added enhanced basic skills training 
for the hard-to-serve students they did 
recruit. Several projects were unrealistic 
in their expectations of clients or their 
assumptions about the needs of employers 
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Prolects providing customized job training 
were more successful in training the proposed 
numbers of students than were other types of 
projects. In some projects, however, the 
customized training subsidized training the 
partner would have done even without the 
project. 

The training provided by most projects was of 
short duration. Except for a few multi-year 
training efforts (of which the grant funded 
oHly a portion, mist students received fewer 
than 100 hours of training. The short 
duration of training reflects the ^act that 
most programs involved some type of custom- 
ized training, which in turn reflects the 18- 
month time frame of the grant. 

1 Staffing shortages in OVAE resulted in 
grantees receiving less technical assistance 
and guidance from project officers than 
initially planned. 

I Most of the third party evaluations of pro- 
jects produced little feedback to grantees 
and limited outcome information at the end or 
the project. 

I Very few of the projects involved students in 
secondary schools. It was not possible to 
determine whether the low representation was 
due to lack of interest or awareness by 
secondary schools or whether it was a result 
of the grant review process. The low 
involvement of secondary schools resulted in 
few lessons about public private cooperation 
at this level. 



Based on these findings and other information collected during the 
evaluation, the study team recommends that: 



Grant awards should be made in the spring 
for project starts in the summer because of 
the greater availability of grantee staff, 
reduced training demands by other programs, 
and more opportunities for planning. 
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The applicant's justification of labor 
needs must address the geographic area 
immediately surrounding (i.e., within 
reasonable commuting distance for the 
students being trained) and use current 
labor market data. 

Applicants must state in detail the role 
and responsibilities of the private part- 
ner(s) and include a statement from the 
partner agreeing to accept this role. 
Applicants should be encouraged to pursue 
only a few partners but have those partners 
much more heavily involved. 

Grantees must be encouraged to request 
changes to the scope of work in response to 
changes in the local labor market and other 
conditions. Some grantees made extensive 
ad hoc changes in the project design while 
others felt they were not allowed to 
deviate from the application. 

The Department should consider awarding a 
six month planning grant combined with an 
18- to 24-month implementation grant. 

Grantees should be held more accountable 
for meeting the objectives stated in their 
application or revised scope. Objectives 
should be stated in operational terms that 
can be measured by the third party 
evaluator. 

OVAE staff should hold additional meetings 
of all project directors as a group. These 
meetings should address management, 
evaluation, and reporting issues. OVAE may 
want to consider a program for ongoing 
technical assistance to grantees, much the 
way other Federal agencies do. 

The Department should formulate procedures 
for ensuring greater participation by 
secondary schools. 
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C. Are Project Costs Reasonable in Relation to Project Outcomes ? 



■ Planning and administrative costs accounted 
for between two percent and 35 percent of 
total project costs, with most projects 
averaging 35 percent. 

■ The cost per student trained and per hour of 
training was similar across projects. 

■ Project costs were greatly affected by the 
monetary value assigned to the equipment and 
other contributions from private partners, 
with no assurance of standard values across 
projects. 



Based on these findings and other information collected during the 
evaluation, the study team recommends that: 



1. Applicants must better substantiate the 
value assigned to in-kind contributions. 

2. The contributions from the private sector 
must be clearly distinguished from 
contributions from the grantee institution. 

3. Grantees should be held to the same non- 
supplanting requirements of other Perkins 
Act programs. 

4. Applicants should be told at the start of 
the grant what cost and performance data 
they will be expected to provide at the end 
of the grant. 
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"Past B— OcMONtnunoN Pbociams 
"Subpart l-Cooperative Oemooitration ProgranM 

"rtOQIAM AUTHOtBKO 

9^?^- . "S»a 411. (a) Prom the amounts available for this nart unrf»p 
o,nt«cuw,th sectional for «j«Aft«l year. thaSe.^ 
20 use 2411. otxu direcUv or through grants to or contractt with State and locd 
/Wi. p. 2479. education^ agenao, postsecondary educational institutiona. ImU- 
tuuons of higher education, and other pubUe and private agenein, 
organizationa. and insUtutions. programs and projecta whiSh tS- 
port— 

"(I) model programs providing improved access to qualitv 

Anu. p. 2450. m se^ 201(b) of this Act and for men and women seeking 

nontraditional occupations; ■«««ng 
"Oi cxamplM of successful cooperation between the private 
sector and pubhe a«uiciM in voeaUonai educaUon. involving 
empioysnor conaortu of emplojrets or labor otganizaUons and 

dengned to demonstrate ways m which vocaUonal eduction 

/.{m «PPrenticeship programs; 

^ JS job training for vocational edu- 

cation^identa which » related to their occupational goals 

SSl^3&i«i«LS «^«~«» laborato? instruftioS 
PnjJjMSd by an eligible reapient; 

(Q placement aervicca in occupations which the students 
are preparing to enter; and 
"(D) where pracUol, projects (such as the rehabilitation 

Sll'iiL'SjS^ econoSl? 

the public 

- 3 E""??^ *° overcome naUonal skill shortages, ss desiir. 
^* J^hSS SLSSffST i^^t of <>)mmerce; and 

Djr Uut Act, and such other neccMry aaaittance ih« ^Ir^f^ 
detjmine. to be necessary for the'^u'Sancmp^etiolrS'tS 

"(2) Not less than 25 percent of the cost of thm Ammisr,m*^n^^ 
progxwns authorized by ^ subpart sSLlS prov&ed h??hHSBr 
ent of the grant or codtnct. andiMbare SS^bTinthe fo™ of 
cash or in.kind contributions, including fSiS^oveS«£ m«S 
nel and equipment ftdriy valued. overnead, peraon- 

"(c) iUl promma assisted under this section shall be— 
^(1) of duect semce to individuals enroUed in such programs: 

"fd) Thi'^^ii Ti? l?"^* providew. 

(d) TheSecretary shall disseminate the resulu of the Drosraraa 
and projeMs asustwl under this section in a aMnerdeSS5?S 
improw the training of teachara,- other instroSoSl SSSneL 
SJSSSSthk aS"^*"" Who are neededrSnS^f 

oSite ^S^.J^n'^.IS^^tSii' 
S-gS^pLSSSSri^ 

and progruns of related instruction). ^ SeawsSSsKlf 

fiS>.?^.i""*^7* coopetmtive^lSS?SrS(S2£S2 
fessibleandappropnata. wMwoerea 

erJc 3' '7 
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PART 412^00PERATIVE 
DEMONSTIATION PROGRAM. 

Src. 

412.1 Whftt U the Coopcniltvr Ocmoastra. 
tlon Program? 

412.2 Who U elltiblft to apply for an award 
under this profcram? 

412.3 What reguiationx apply to this pro- 
tram? 

412.4 What definitions apply to this pro- 
tram? 

. $t<f«f«fy Attltf UiHl«r Thit Pr««r«m? 

412.10 What types of projccU may be 
funded? 

412.11 How does the Secretary establish 
priorities for this protram? 

Svb#«rt C-4R«t«rv«4| 

$vbfi«H D*-H«w 0—% th« S«cr«t«ry M«li« • 
Grant? 

412.30 How does tlie Secret nry evaluate an 
application? 

412.31 What selcrt ion criteria itoes the Srr- 
retary lue? 

Sithpmn i^Whaf C«iHliHMt Mwtt M M«t by • 

412.40 What cost sharint requirement is 
im;>osed under thU protram? 
AtrrKoaiTY: Sec. 411 of the Carl D. Per 
kins Vocational Education Act. 20 U.S.C. 

2411. rts enacted by Pub. L. 9R 524. unie^ 
otherwise noted. 

SoitacE: 50 FR 33260, Aua. 16. 1985. un\t%% 
otherwise noted. 



Subport A— G«n«rot 

« 412.1 What Ik the <»operulive Demon. 
Mratiim Proitram? 

(a) The Cooperative Detnonstrnlion 
Program provides fitiandnl nssistance 
for— 

(1) Model projects providing Im^ 
proved access to qttallty vocational 
education programs for certain indl- 
viduals: 

(2) Projects that are examples of 
successful cooperation between the 
private sector and public agencies In 
vocational education: 

(3) Projects to overcome national 
skill shortages: and 

(4) Other activities which the Secre- 
tary may designate that are related to 
the purpose of the Act. 

(b) ProJecU eligible for assistance 
are described In { 412.10 

(Authority: See. 4lha»: 20 li.S.C. 24lhan 



14112 Who Is eligible to .pp|y 
•ward under ihU program? 
The following are eligible to apply 
for an award under this program- 
(SEAs>.^^^^ educational agencies 
(i2as> educational agencies 

(c) Postsecondary educational Instl- 
tutlons. 

(d) Institutions of higher education. 
<e) Other public and private agen- 

cies. organizations, and Institutions, 
(Authority: Scc.4U(ai; 20 U.S.C. 24rKa)) 

f 1 12.3 What reitulali»mt apply to thin pro* 
rram? 

The following regulations apply to 
the Cooperative Demonstration Pro- 
gram: 

40^0 ^ regulations in 34 CPR Part 

(b) The regulations in this part, 
(Authority: Sec. 4M: 20 U.S.C. 2411) 

112.4 What defmltlomi apply to (hiN pro. 
gram? 

The definitions In 34 CPR 400.4 
apply to the Cooperative Demonstra- 
tion Program. 

( Authority: Sec. 4 U: 20 U.S.C. 24 U ) 

SubfMrt »*-Whart Kindt of Activifkt 
Oo«» tht Stcrofary Assist Undor 
This Progrom? 

» 412.10 What type* of projeeu may be 
funded? 

(a) The SecreUry may support di- 
rectly or through grants, cooperative 
agreements, or contracts the following 
types of proJecU: 

(I) Model proJecU providing im- 
proved access to quality vocational 
education programs for— 

(I) Handicapped Individuals: 

(II) DlsadvanUged Individuals: 

(lit) AdulU who are In need of train- 
ing and retraining: 

(Iv) IndlvlduaU who are single par- 
ents or homemakers: 

(V) IndlvlduaU who participate in 
programs designed to eliminate sex 
bias and stereotyping In vocational 
education: 
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(vl) CrtmiAal offenders who are serv- 
ing in a correctional instUuUon; and 

(vU) Men and women seeking non- 
traditional occupations. 

(2Ki) ProJecU that are examples of 
successful cooperation between the 
private sector (Including employers, 
consortia of employers* labor organixa* 
tlons« and building trade councils) and 
public agencies in vocational educa- 
tion, including 8Ute boards and eligi- 
ble recipients* 

(11) The projects described In para- 
graph Ca)(2Ki) of this secUon must be 
designed to demonstrate ways in 
which vocational education and the 
private sector of the economy can 
woric together effectively to assist vo- 
cational education students to attain 
the advanced level of sidlls needed to 
make the transition from school to 
productive employment, including*- 

(A) Work experience and apprentice- 
ship projects; 

(B) Transitional worksite Job train- 
ing for vocational education studer.ts 
which is related to their occupational 
KoaU and closely linked to classroom 

and laboratory iiustructloti provutt it i»\ 
an eligible recipient: 

(C) Placement services In occupa- 
tions which the students are preparing 
to enter, and 

(D) Where practical, projects tliat 
will benefit the public such as the re- 
habilitation of public schools or hous- 
ing in inner cities or economically de- 
pressed rural areas* 

(Hi) The projects described in para- 
graphs (a)(2) (i) and (11) of this section 
may include Institutional and on-the- 
job training, support services autlior- 
Ized by the Act, and such other neces- 
sary assistance as the Secretary deter- 
mines to be necessary for the success- 
ful completion of the project. 

(3) Projects to overcome national 
skill shortages, as designated by the 
Secretary in cooperation with the Sec- 
retary of Labor, Secretary of Defense, 
and Secretary of Commerce. . 

(4) Such other activities which the 
Secret&ry may designate which are re- 
lated to the purposes of the Act. 

(h) All projects assisted under the 
Cooperative Demonstration Program 
must be^ 

(1) Of direct service to the Individ- 
uals enrolled: and 

(2) Capable of wide replication by 
service providers. 

(Authority: Stc. 411 (a), (t». <c): 20 U.S.C. 
2411 (a>,(bn (€» 



0 412.11 How docs the Secretsrr eatsbllnh 
pHoridct for this progtaia? 

(a) The Secretary may announce 
through one or more notices published 
in the FcDOUL Rccxsm the priorities 
for this program (including any na- 
tional skill shortages to be addressed) 
if any, from the types of projects de- 
scribed in 1 412,10. 

(b) The Secretary may establish a 
separate competition for one or more 
of the priorities selected. If a separate 
competition is established for one or 
more priorities, the Secretary may re- 
serve all applications that relate to 
those priorities for review as part of 
the separate competition, 

(Authorttr Sec. 411; 20 U^.C. 2411) 
Subport C-4Reiervod] 

Subpart 0— How Ooos tho Stcrotory 
Moko m Gront? 

6 4I1S0 How doM the Seercury evaluate 
an application? 

<a) The Secretary evaluates an appli- 
cation for . a grant or cooperative 
agreement on the basis of the criteria 
in I 412.31. 

(b) The Secretary may award up to 
100 points, including a reserved 15 
points to be distributed in accordance 
with paragraph (d) of this section, 
based on the criteria in I 412.31. 

(c) Subject to paragraph (d) of this 
section, the maximum possible points 
for each criterion is indicated in pa- 
rentheses after the heading for each 
criterion. 

(d) For each competition, as an- 
nounced in a notice published in the 
Feocral Register, the Secretary may 
assign the reserved 15 points among 
the criteria In 1412.31. 

< Authority: Sf c. 4 1 1 : 20 U.S.C. 24 1 1 ) 

< Approved by the Office of Manaxfincnt 
and Budcci (tnder control number 1830- 
0013) 

S 412.31 What ftelecUon criteria doen the 
Secretary ute? 

The Secretary uses the following se- 
lection criteria in evaluating each ap- 
plication: 

(a) Need. (15 points) 

<1) The Secretary reviews each ap« 
plication for Information that shows 
the need for and the soundness of the 
rationale for the project 
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(2) The Sccreury looks for Informt- 
tion that shows-* 

<l) A clear description of the need 
for the proposed project; 

ni) Specific evidence of the need for 
the project: 

(HI) A description of any ongoliiR 
and planned activities In the communis 
ty relative to the need. Including, if 
appropriate, the relationship of any 
local, regional or State economic do- 
volopinent plan: 

(iv) Evidence tliat drnionsiratcs the 
vocational I raining to be provided is 
ciesigned to meet current and project- 
t <l ocrupat tonal needs: 

(V) A clear statement of what the 
project seeks to demonstrate; and 

(vi) Evidence that the project Is 
Hkely to serve as a model in the 
future. 

(b) Plan o/opemtioTu (20 points) 

(1) The Secretary reviews each ap. 
plication for information that shows 
the quality of the plan of operation 
for the project. 

(2) The Secreury looks for Informa. 
tlon that shows*- 

(l> High quality In the design of the 
project; 

(II) An effective plan of management 
that ensures proper and efficient ad- 
ministration of the project: 

(lii) A clear description of how the 
objectives of the project relate to the 
purpose of the program: 

(iv) The way the applicant plans to 
use Its resources and personnel to 
achieve each objective: and 

(V) A clear description of how the 
applicant will provide equal access and 
treatment for eligible project partici- 
pants who are members of groups that 
have been traditionally underrepre- 
sented. such as— 

(A) Members of racial or ethnic mi 
nority groups; 

(B) Women; , 

(C) Handicapped persons: and 

(D) The elderly. 

(c) Quality of key personnei (10 
points) 

(1) The Secretary reviews each ap- 
plication for information that shows 
the qualifications of the key personnel 
the applicant plans to use on the 
project. 



I*" OumncMlons of euh of the 
(111) The time that each person re- 

plijek- 'aS/*"""" """"'^ ^° thS 
JlV *^''* '"^'"t to Which the appli. 
cant, as part of lu nondiscriminatory 
employment practices. enwuSgw ao^ 
Plications for employment from p*?. 
sons who are members of grou j that 

(A) Members of racial or ethnic mi- 
nority groups: 

(B) Women: 

(C) Handicapped persons: and 

(D) The elderly. 

(3) To determine personnel qualifi. 
cations, the Secretary considers expe- 
rience and training. In fields related to 
the objectives of the project, as well as 
otn^Mnformatlon that the applicant 

(d) Budget and cost effectiveness. (10 
points) 

(1) The Secretary reviews each ap. 
plication for Information that shows 
the project has an adequate budget 
and is cost effective. 

(2) The Secreury looks for informa- 
tlon that shows*<- 

(I) The budget for the project Is ade- 
quate to support the project activities: 
and 

(II) Costs are reasonable in relation 
to the objectives of the project. 

(e) Evaluation plan. (5 poInU) 

(1) The Secretary reviews each ap- 
plication for Information that shows 
the quality of the evaluation plan for 
the project. 

CKoss-ficrc»nirc. S«» 34 CPR IS 590 
(Evaluation by the trtnlee>. 
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(2) The Secretary looks for Informa- 
tion that shows methods of evaluation 
that are appropriate for the project 
and. to the extent possible, are objec* 
live and produce data that are quanti- 
fiable. 

(f) Adequacy of resources. (5 points) 

(1) The Secretary reviews each ap- 
plication for information that shows 
that the applicant plans to devote ade- 
quate resources to the project. 

(2) The Secretary looks for informa- 
tion that shows— 

(I) The facilities that the applicant 
plans to use are adequate: and 

(ID The equipment and supplies that 
the applicant plans to use are ade- 
quate. 

(g) Private sector involvemenL (5 
points) 

(1) The Secretary reviews each ap- 
plication for information that shows 
the involvement of the private sector. 

(2) The Secretary looks for informa- 
tion that shows-- 

(i) -Private sector involvement in the 
Planning of the project; and 

(ii) Private sector involvement in the 
operation of the project. 

(h) Employment opportunities, (5 
points) 

The Secretary looks for Information 
and documenution of the extent to 
which trainees will be employed in 
Jobs related to their training upon 
completion of their training. 

(I) Dissemination. (lO^polnts) 

(1) The Secretary reviews each ap. 
plication for Information that shows 
that the applicant has an effective and 
efficient plan for disseminating Infor- 
mation about the demonstration 
project. Including the results of the 
project and any specialized materials 
developed by the project. 

(2) The Secretary looks for informa- 
tion that shows— 

(1) High quality in the design of the 
dissemination plan and procedures for 
evaluating the effectiveness of the dis- 
semination plan: 

(ii) A description of the types of ma- 
terials the applicant plans to make 
available and the methods for making 
the materials available: 

(Hi) Provisions for demonstrating 
the methods and techniques used by 
the project; 

(Iv) Provisions for assisting others to 
adopt and successfully Implement the 
project or methods and techniques 
used by the project: and 

(V) Provisions for publicizing the 
findings of the project at the local. 
State, or national level. 

< Authority: See. 4 11 : 30 U.S.C. 24 1 U 
(Approved by the Offit'? of Manaacmnit 
and Budgfi under control mtml>er 1830- 
0019) 



SubpoH E^Whof Conditions Mutt B« 
M«t by o R«cipi«nt? 

9 412.10 What tout nhaHng requirement tn 
tm^NMed under thin program? 
(a) A recipient shall provide not less 
than 25 percent of the cost of the 
demonstration project it condticis 
tmder this program. 
J^Jji accordance with Stibpart G of 
34 CPR Part 74. the non-Federal share 
may be in the form of cash or in kind 
contributions, including the fair 
market value of facilities, overhead 
personnel, and equipment. 
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FY 1988 GRANTEES REVIEWED IN THE EVALUATION 



Grantee 



- reject 



Central Community College- 
Platte Campus 



Competency-Based Modular Assessment 
and Training for Maintenance 
Technicians in Manufacturing 



Division of Vocational 
Education Services 
State Department of Education 
Montgomery, Alabama 



Student Apprenticeship Linkage 
in Vocational Education 



El Paso Community College 
El Paso, Texas 



CAREER Program: Career Assessment, 
Remediation, Education, 
Employment, and Re-entry 



Francis Tuttle Vocational 

Technical Center* 
Oklahoma City, Oklahoma 



High Technology Partnership 
Project 



Greenville Technical College 
Greenville, South Carolina 



Project TEAM: Technical 
Education Advancement Modules 



Hampden County Employment 
and Training Consortium 
Springfield, Massachusetts 



Project CREATE: Cooperative 
Resources to Enhance Access to 
Jobs Through Technical Education 



Indian Hills Community College 
Ottumwa, Iowa 



Indian Hills Cooperative 
Demonstration Program 



Indiana University of 

Pennsylvania** 
Reschini House Indiana, 
Pennsylvania 



Northwestern Pennsylvania 
Cooperative Demonstration for 
Technical Updating 



*responded to telephone survey 
•received site visit 
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Northampton Community College** Turn-key Surface Mount 



Bethlehem, Pennsylvania 



Training Program 



Parkland College 
Champaign, Illinois 



Advanced Certification Program 
for Computer Graphic Specialists 



Portland Community College 
Portland, Oregon 



Women in Education for 
Apprenticeship and Non-Traditional 
Employment 



Postsecondary Vocational - 

Technical Education 
Concord, New Hampshire 



New Hampshire Automotive 
Education Collaborative 



Richland School District** 
Kennewick, Washington 
Future 



Materials Technology: The Common 
Core Skills That Are Shaping the 



Skyline College** 
San Bruno, California 



Toyota/Skyline Partnership 
for Automotive Technician 
Training 



Southern Growth Policies 
Board 

Research Triangle Park, 
North Carolina 



Consortium for Manufacturing 
Competitiveness 



Toledo Public Schools 
Toledo, Ohio 



Industrial Automation Mechanic 
Model Curriculum 



University of North Dakota- 
Lake Region 
Devils Lake, North Dakota 



Flight Simulator Maintenance 
Technician 



University of Wisconsin* 

State** 
Menomonie, Wisconsin 



Implementing a High-Tech Training 
Model for Rural Based Business 
and industry. Technical Colleges, 
and Local and State Education 
Agencies 



*responded to telephone survey 
*received site visit 
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Valencia Community College** 
Orlando, Florida 



Valencia Community College** 
Orlando, Florida 



A Model, Repli cable Advanced 
Manufacturing Demonstration 
Project 

Film Production Technology 
Training Program 



Ventura Community College 

District** 
Moorpark College 
Moorpark, California 

Waubonsee Community College 
Sugar Grove, Illinois 



Non-college Bound Student 
Demonstration Project- 
El ectroni cs/Laser/El ectro-opt i cs 



A Comprehensive Development Plan 
in Office Skills 



Yakima Valley Community 

College 
Yakima, Washington 



Extending Health Training and 
Services to Rurally Isolated 
Populations in a Depressed Area 



♦responded to telephone survey 
•received site visit 
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FY 1989 GRANTEES REVIEWED IN THE EVALUATION 



Grantee 



Project 



Bronx Community College** 
Bronx, New York 



A Model Program Demo*"strating 
A Public/Private Sector 
Cooperative Training Program 
in Radiologic Technology 



Center for Occupational 

Research and 

Development (CORD)** 
Waco, Texas 



Semiconductor Manufacturing 
Technology Program 



Clackamas Community College** 
Oregon City, Oregon 



Precision Manufacturing 
Technology Project 



Fox Valley Technical College** 
Appleton, Wisconsin 



Hampden County Employment 

and Training Consortium** 
Springfield, Massachusetts 



A "Systems" Approach to Providing 
Cost-Effective Training and 
Technical Assistance in CIM to 
Small Manufacturers 

Project: High-Tech '90 



Home Builders Institute** 
Washington, DX, 



Training and Certification 
Program for Smart House 
Instal 1 er/Technici ans 



Howard Community College*^ 
Columbia, Maryland 



Enhanced Re-entry Nurses 's 
Education Using Interactive 
Videodisc Assessment/Instruction 



Indian Hills College 
Ottumwa, Iowa 



A Model to Provide Degree-bearing 
Training to Non-traditional 
Students 



*responded to mail survey 
*received site visit 
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Illinois Eastern Community 

College** 
Olney, Illinois 



Training in Semiconductor 
Technology 



Indian Hills Community** 

College 
Ottumwa, Iowa 



Demonstration Model Project 



LTV Steel Company** 
Whiting, Indiana 



Electronic and Instrument Repair 
Technician Apprenticeship 



Luzerne County Community 

College** 
Nanticoke, Pennsylvania 



Advanced Technology Center 
Computerized Numeric Control 
Cooperative Demonstration 
Training Program 



Nebraska Department of 

Labor** 
Lincoln, Nebraska 



Rural Allied Medical Business 
Occupations Program 



North Clackamas School** 
District #12 
Milwaukee, Oregon 



Student Cooperative Training 
Units Program 



Partners for American Business-Education Venture for 

Vocational Education (PAVE)** Health Care Occupations Training 
Alexandria, Virginia 



Southwestern Community** 

College 
Chula Vista, California 



Comprehensive Aerospace 
Manufacturing Technology 
Program 



State Center Community** 

College District 
Fresno, California 



Advanced Technology Center 



Waubonsee Community College** 
Sugar Grove, Illinois 



Survival Skills for Office 
Technicians 
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West Virginia Department 

of Education*^ 
Charleston, West Virginia 



Cooperative Demonstration of 
High Technology Programs in 
Computer Language and Health 
Occupations 



West Virginia Northern 
Community College** 
Wheeling, West Virginia 



High Technology Training: The 
Other Half of the Equation 



♦responded to mail survey 
•received site visit 



ERLC 



220 



Appendix C-1 

RAW SCORES OF PROJECT LOGIC FOR FY 1989 PROJECTS 



221 



c-1 



CM 



ERIC 



o 

X 

•o 
c 

(D 
Q. 
Q. 
< 



CO 

»- 
o 

LU 

o 

CC 
Q. 

o> 

00 

> 

CC 

o 

LL 

g 

o 
o 

»- 
o 

HI 
—3 

O 
CC 

CL 
LL 
O 

CO 
LU 
DC 
O 

o 

CO 



< 

DC 



E 
3 



CO (0 

E 
3 



(0 



tr 

a 

z CO 
tr 

8. 

CX 
=> 
CO 



tr 

a 

tr 
cx 

CO 



2 E o 
a> 5> E 

PI 



o E « 

^ k- Q) 

E 

zcoO 



2co 



a> 

XJ CO 

II 



a> 

0) 

c 

E5 
O 



GO 



CO 



COCNJt;.CNJC\JCNJCO^CJ^CNJ^^ 



CO 



to 

X 



o 
c 



o 

o 



o o 



O-t-CVJ-t-COOOO^OT-OOO 

+ + + + + + 



CO 

+ 



CNJ 



+ + + + + + 



1- 10 
+ + 



O O <D 



lOCD^^COCOCDCOCO^OO 



co^io^'^cMco'^in'^^^'^co 



00 CO ^ 



LOOOCDCOO^GOOr^CD 



m^COlDOO^COCD^CDCDtOlOCO 



CO 



CD 
CD 

O 2 



^ 1 

E J~ 

is ® 

< m 



a> 

o 
o 

c: 

E 

E 
o 
O 



S 

CO 



o 

o 
55 

(0 
CD 

o 
o 
c 

03 

O 



f 

O 

O 



3 
E 

E 
o 
O 

CO 

£ 
01 

u 
« 

O 



CD 

© 

O 
O 



(0 

« 
5 

E 
_2 
o 
O 



Q 
CC 
O 

o 



Oi 
O 

o 
O 
>, 
*c 

E 
E 
o 
O 

o 

(/) 

CO 
CL 



(/) A O 

§ I o 
^ t 



o 
a> 

05 ~~ 
> 
X 

o 

Li. 



o 5 
O -o 



E 
£ 
o 



X 



3 O 
CD 'O 
9 to 

£ $ 
o 
X 



© 
"o 
O 



c 
O 
*o 



CD 

o 
O 
c 

B 

to 

OJ 

lU 

(O 

0 



o 
X 



CD 

O 

CO 



CD 
O 

O 

c: 
o 



CO 
CL 



CM 



c-2 



g> CO 

3^ 



5' 

E 



i 

a> 



of 
E 



T5 
O 

C 

c 
o 
o 



O 

X 

c 
o 

CL 
CL 
< 



CO 



a 

Q. 

:co 
tr 

a 

Q. 
CO 



CO 



tr 

a 

Q. 

:co 
tr 

& 

Q. 
CO 



o E 25 
o 5J E 

E 

Z Jo 



o g g 
E 

X) ^ o 
zcoO 



2 CO 
Q).S> 

il 



Q> 

o to 

it 



E 
2 
O 



IT) CO CO UT) IT) 



rj-CNi^Up'^CpCOl^Cp 



CNJOOOOOOi- 
+ + 



CO 



CO xj- CO 



+ + + + + + 



O^OCNiOOCycy 



CO 



+ 



CDCDOCOCaOOOCO 

• -1- T- C\J ^ + 

+ + + + + 



CO LO <£> in 

T- + + + + 



in ^ CO CO 



m CO 



inr^omcNjoooo 

^ ^ ^ C^i t- CM 



CNJ O rj- 00 00 



Gococococvjinoco 



CO O CO CO o 



CO CO 



in 



CO 



o 
O 

o 



O 

O 

>s 

C 
3 

E 
E 
o 
O 

o 

c 

w 
(SI 



t: 

CO 

Q. 

a 

o 

CO 

(0 
CO 

2 
o 



o 

o 



E 

E 
o 
O 
c 
o 

E 
CO 

JZ 

t: 
o 



O 

o 
x: 

.55 

(A 
CO 

E 

fO 

o 

JCO 

O 

JZ 

tr 
o 



ai 
> 
< 



CO 

b 

o> 

o 
O 
c 

w 

B 

CO 

o 

O 

CO 



CO 

b 

f 

o 
O 
i— 

S 
c 
0) 

O 

fo 
55 



lege 


lege 


lege 


Col 


Col 


Col 








Communi 


Communi 


CommunI 


CO 


CO ^ 


5 


'o 


o S 




Valer 
(CIM] 


Valer 
(Heal 


Valer 
(Tele 



"o 
O 

5k 

'c 

3 

E 
E 
o 
O 

o 
o 

CO 

c 



3 
CO 



tr 

(0 
CL 



m — ' 



5 5ii 



o 
O 
c 

k. 
0) 

t: 
o 



> 

to 

5 



CO 



Appendix C-2 

RAW SCORES OF PROJECT PLAUSIBILITY FOR 
FY 1989 PROJECTS 



226 



ERIC 



c-3 



> ^ 

•C CO 

a. o 

CO 

1°- 



+ + + + + + + + + + + + + + + + 



•n o 
S o 



(^5 



(D 

C CO 

*o < 



CO 



2 o 
*t: CO 

WJ rr 



CM O ^ 



cy cy 



CO 

+ 



CO 

+ 



T~ Tj- CO CD 
+ t— + 

+ + + + 



CO 



CO 



00 o 



CO 

+ ^ 
+ 



o ^ 



1" o 



'J t t 



CD 

+ 



CO CO 



CO 



CO (A (A 
+ + + + 



r-* O ^ 



o 



o 



CD 

+ + 



ir> CM 
+ + 



CD 

+ 



? ? 



0 

o 
O 



CO 

E 

(0 

JO 
JO 

< 



CO 

c 



c 
CQ 



o 

o 



E 
E 
o 
O 

X 

c: 
o 
I.. 

GQ 



o> 

J) 

*o 
O 
o 
CO 

55 

CO 

o> 
o 
o 
c 

CO 

ti 

CO 

O 









0) 


o 








O 








nity 


lege 




Col 


3 


o 




>s 


E 


O 






E 


c 




3 


o 


!se 




E 


O 




E 


(/> 


CD 




o 


CO 


bia 




O 


E 




o 


lacka 


olumi 


ORD 


IPas 


o 


o 


o 





o 



a> 



t: 
o 

CO 

c 
o 
O 
>» 
c 

3 
O 

O 
c 
0 



(0 

X 



5 

3 



S 

3 

CD 



O 

X 



O 

o 



E 
E 
o 
O 

'B 

CO 

o 
X 



0 

i 

o 
O 



c 
a> 
O 

(O 

o 



o 

O 



o 

*<0 
CO 

LU 
i/> 
O 



O 
O 



c 

3 

E 
E 
o 
O 

(O 



CO 

= = ^ 



0) 

O 
c 
o 

if) 



CO 

a. 

c 
o 



(0 

a. 



O 

•o 

s 

c 

c 
o 

o 



c-4 




0) 
» 

c 
o 
O 

O 



T5 
C 

CL 
Cl 
< 



1^ *r 



0. 

X> 



2 o) 
«.S 

to c 



.£ 0) 



(^2 



o 

o o 
*o < 



CO 



c 
CO 



1 

w 

CL 



^ T ? 



CVJ o 



CVJ T- T- CVJ O T- 



CVJ 



CNJ 

+ 



W CO <0 CO CVJ 

+ + + 



00 0> O OP 



- + 



0000000 -r- 



+ 



? ? t 



O) 



in o <p cj o <D cj 



'^CpO<NiT;-0)<Nil>'^'7^?^ 



CM 

+ 



u, « ^ « CJ CJ CJ 



CM 

+ 



CM 

+ 



CO 

+ 



CM ^ 

+ + 



in 
+ 



o 

o 
■5 
B 

55 



0 

o 

o 



E 
E 
o 

o 

c 

Q> 
N 
3 



tr 
® 

o 

o 
-J 

if) 
CO 

o 
2: 



f 

o 

o 



E 
E 
o 
O 
c: 
o 

E 
o 

2: 



J2 
o 
o 

0) 



o 
iS 
O 

o 
2: 



LU 
> 
< 

a. 



o 

b 

o 

o 
O 

E 
© 

o 

CO 



J2 
O 

o 
O 

c 
O 

T5 
So 



1 

o 

CD 

o 
O 



E 
E 
o 
O 
<g 
o 
c 

m 
> 



X P 



Of) 

o 

o 
O 



3 
E 
E 
o 
O 
n 
o 
c 

m 
> 



CD 
O) 
O 

O 

O 
>* 

"E 

3 

E 
E 
o 
O 
n 

o 
c 
o 
m 
> 



o o 



E 
E 
o 
O 
o 

CD 
CA 
C 



OL 



3 
(0 



o 

3 

5 uj 



o ■ 



o 
O 
c 

sz 
tr 
o 
z 

'c 



CO 

a> 
5 



ERLC 



Appendix D 

BRIEF DESCRIPTIONS OF FY 1988 GRANTEES 
SURVEYED OR VISITED 



231 



D-1 

Richland School District, Kennewick. Washington 

The original plan, largely implemented, was to introduce a one-year 
course in materials science and technology (MST) to seven high schools 
and one community college. The MST course curriculum had already been 
developed by a teacher at Richland High School with support and 
technical assistance from Battelle Northwest Laboratories (the private 
partner). The course was a 180-hour, hands-on, science and vocational 
course about glass, ceramics, metals, composites, and wood. Two 
teachers from each of the seven high schools were trained in a three 
and a half week summer workshop. They adapted MST outline and workbook 
to their own school and local needs, and trained 237 students during 
the 1989-90 school year. One site dropped out at the mid-year point 
because of school construction and staff turnover. 

Northampton Community College, Bethlehem, Pennsylvania 

The National Training Center for Microelectronics at NCC proposed 
to providing local manufacturing companies (the private partners) with 
customized job training in surface mount technology (SMI). During the 
grant period, the Center expanded its existing training program in SMT, 
enhancing four existing courses, creating seven new courses, acquiring 
new equipment, and producing a national teleconference. The project 
trained 233 employees of seven microelectronics firms during the grant 
period at reduced or no cost to the companies. The project also 
reached an estimated 2000 employees at 18 locations nationwide through 
two four-hour teleconferences, one of which was interactive. The 
subject of the conferences was "Packaging in the 1990s," and videotapes 
of the teleconference were sold to private companies and donated to 
other universities. The project was generally implemented as proposed, 
but trained only about half as many students as planned because 
companies enrolled fewer students than predicted. 

Valencia Community College, Film, Orlando. Florida 

The VCC staff established as proposed film production technology 
program to train students for jobs in the growing local film industry. 
With technical support from Universal Studios (the private partner), 
VCC developed the program* s curriculum and implemented a 15-week course 
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offered three times during the grant. A total of 135 students were 
trained in stagecraft, sound, set construction, camera/editing, and 
post production. Together, the three sessions produced a full-length 
feature film entitled ••Sealed With A Kiss.'' The project was 
implemented as outlined in its proposal. 

Valencia Community College. Manufacturing. Orlando, Florida 

VCC proposed providing a local manufacturing company with 
customized job training in automated manufacturing technology. VCC was 
already working with Stromberg-Carlson, Inc. (the private partner) 
under <i State of Florida grant to help Stromberg-Carlson introduce high 
technology-based manufacturing processes. During this project, VCC 
trained 565 Stromberg-Carlson employees with the curriculum designed 
under the earlier grant. A total of 26 classes were offered in 17 
different courses; the courses averaged 65 student contact hours. The 
project was implemented as proposed, although 30 percent of the 
students dropped out prior to graduation due to other demands on their 
time. 

University of Wisconsin - Stout. Henomonie. Wisconsin 

The project proposed implementing a ••model •• high technology 
training program in three technical colleges and 12 high schools in UW- 
Stout's service area. Project staff conducted an initial four-week 
summer workshop and trained teachers from the participating secondary 
schools and postsecondary technical institutes how to conduct local 
needs analyses and to develop module curriculum. Teachers at each 
school interviewed local industry officials to determine high-tech 
training and skill needs and to develop course modules. Teachers spent 
the 1989-90 school year preparing their module(s) and testing them in 
classes. The project planned to complete 41 modules— ranging from a 
few hours to a full semester of instruction. The project was 
implemented as proposed except that teachers at the participating 
technical colleges did not have time to begin training employees of the 
private sector partners. 
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Skyline Community College. San Bruno. California 

The vocational division at the college proposed a joint training 
program for service technicians with Toyota Motors Sales (the private 
partner). The program, called T-TEN, included 16 weeks of formal 
instruction per year for three years and part-time work at a reduced 
wage in Toyota dealerships. Toyota provides financial incentives to 
the college and to students for implementing and graduating from the 
program. The project was underway, with state finance, prior to the 
Cooperative Demonstration grant. Potential students were recruited 
through newspaper ads and selected by dealership personnel during an 
annual meeting. The college began the process of obtaining national 
certification for its automotive program (NATEF). At the end of two 
years, a total of 17 students were enrolled in vhe program and two 
Skyline instructors had received extensive Toyota training and 
developed the training curriculum using Toyota materials. The project 
was largely implemented as planned, except 1) there were fewer 
participants because there were fewer job opportunities with Toyota 
dealerships, and 2) most students ended up working full-time because 
they needed the money and dealers needed the staff. 

Ventura Community College District - Hooroark College. Moorpark. 
C alifornia 

The original plan was to establish a coordinated high school- 
college program in electronics and laser/electro-optics for at risk 
students in 8 high schools. In addition, the local business/labor 
council (BLC~the private partner) would arrange for field trips to 
potential employers, transportation among sites, counseling, and other 
activities. The project curriculum was to be developed at the college. 
The initial plan was modified considerably over the grant period. A 
summer remedial basic skills program was not held. High school 
instructors received informal training from the college as needed to 
implement the program. Planned inter-school transportation was 
simplified. The BLC role diminished substantially. As jobs in laser- 
optics decreased, college attendance became a more likely student 
outcome. In the end, however, the schools adopted the college- 
developed program, 52 students from 8 high schools completed the 
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coordinated instructional program, more are now enrolled in the 
college. Students received instruction at 4 cf the schools for 4 days 
a week and at Moorpark one day a week« 

Indiana University of Pennsylvania. Indiana. Pennsylvania 

The initial goals of this project were anibitious~co establish 
collaborative technical training between Indiana U, county vocational 
schools, other postsecondary institutions, private trade schools, 
regional economic development agencies and the private sector. 
Problems in coordinating with the regional economic development agency, 
scaled back goals. In the end, the project delivered a variety of 
short--term training programs to the employees of smaller manufacturing 
and other firms in the area (the private partners) at no cost to the 
companies or employees. Courses varied from basic math to the use of 
sophisticated computer controlled machinery. Most courses were offered 
by the county vocational schools and approximately 648 students 
received training or attended product demonstrations. Thirty-three 
classes were organized. 

Francis Tuttle Vocational-Technical Center. Oklahoma City. Oklahoma 
Based on telephone survey only: The goal of this project was to 
increase enrollment in the High Tech Center within the institution by 
providing academic remediation to adults who would not otherwise 
qualify, and to extend formal instruction with internships. This 
project was largely implemented as planned. A recruitment campaign was 
undertaken and a self-paced learning lab installed. To attract 
students, the project provided tuition reimbursement for 220 students 
without regard to financial need. Eighty percent of the Center's 
students used the lab. Internships with stipends paid from the grant 
were provided to 20 students (sites were the private partners). The 
project recorded a substantial increase in enrollments and a dropout 
rate of 30 percent. 
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Bronx Community College 
Bronx, New York 

Bronx Community College (BCC), one of the City University of New 
York (CUNY) Colleges, Is located In the hub of urban activity. The 
Radiologic Technology (RT) Program, funded by the Cooperative 
Demonstration Program grant, was administered through the Physics 
Department at BCC. 

The goals of the program were to provide students with employment 
opportunities and to provide hospitals and other medical facilities 
with trained radiographers In order to Increase their employee pool. 
More specifically, as outlined In the proposal, the program was to: 

■ help alleviate the current shortage of 
radiologic technologists at Columbia- 
Presbyterian Hospital; 

■ help Improve the recruitment of Blacks 
and Hispanlcs Into radiologic technology; 
and 

■ ajgresslvely recruit new workers from 
nontradltlonal labor pools Including the 
unemployed, recent Immigrants, and older 
workers. 

The work of the partnership would also Involve conducting 
placement tests, arranging for student stipends, offering student 
advisement, arranging for the program to be accredited, preparing the 
trainees for licensure, assisting Informally In placing students In 
jobs, and preparing a manual documenting the BBC training model for 
dissemination. 

BCC teamed up with Columbia-Presbyterian Hospital, as planned.. 
After the grant was obtained a second partnership, with Monteflore 
Hospital, was formed. The hospitals and BCC are all located In New 
York City. The partnership began with the Cooperative Demonstration 
Grant. A hospital workers union, Hospital League 1199, was also a 
silent partner, granting a tuition loan to union members already 
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employed at hospitals who became RT students. The loan w111 be 
forgiven If students work for two years and three months as a union 
member in the field of radiology. 

The role of the hospitals was to provide x-ray facilities and 
related equipment and supplies for clinical experiences; in-service 
education; emergency health care services; and free lunch and parking 
for the students. 

Partner meetings took place about once a month. Instructors In the 
program had previous ties with both hospital partners, therefore 
informal contact between BCC and the partners was maximized. 

The head of the Physics Department at BCC served as the Project 
director for the RT Program and a program director and two instructors 
were hired to teach RT courses. The instructors were hired before the 
program began. The program director and instructors were experienced 
in the field of Radiologic Technology training and brought with them 
curricula and course syllabi, which were later revised. 

This program intended to serve Blacks and Hispanics, persons with 
handicaps, and older workers on a larger scale than actually occurred. 
Because the program was only 18 months long, it did not meet the 24- 
month requirement put forth by the State of New York. The State agreed 
to allow students to get credit for an 18-month program only if they 
had previous college experience. Twenty-one students were enrolled at 
the beginning of the program, however only 11 were still enrolled by 
the program's final stages. These included three white Americans and 
eight immigrants, of which three were white (Russians). 

Students participated in the program for 40 hours per week. This 
included didactic training, which took place at BCC, and clinical 
training, which took place at the hospitals two to three times a week 
from 8:30 a.m. to 4 p.m., usually by an RT instructor. Grant money was 
used in part to pay for students' tuition and a stipend. Union 
employees received an additional stipend. 

Support services were another aspect of the program. Special 
needs were identified for Immigrant students having difficulty with 
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English, and tutors were hired. As the program went on, students with 
academic problems received tutoring as arranged by the department. 

The program has continued beyond the life of the grant, with a 24 
month RT associates degree program institutionalized and the continued 
involvement of a11 partners. Accreditation was received from the Joint 
Review Committee on Educational Programs in Radiologic Technology. 

The project was originally intended to develop a manual for use by 
institutions wanting to replicate the model, and a workshop was also 
proposed. The manual is currently in progress. Dissemination to all 
NYC/CUNY colleges took place. The dissemination of the program was 
widespread, mainly through local newspaper articles, the CUNY press, 
and in RT IMAGE, the major trade journal of the profession. 
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Center for Occupational Research and Development (CORD) 

Waco, Texas 

The Center for Occupational Research and Development (CORD), 
located In Waco, Texas, Is a private, nonprofit organization that 
specializes In science and technical curriculum writing and whose 
purpose Is to help educators address the technical education needs of 
workers. 

CORD used Its Cooperative Demonstration Program grant to support 
the Semiconductor Manufacturing Technology (SMT) program. The goals of 
the program were to Identify skills and develop a curriculum that 
satisfied the needs of the semiconductor manufacturing Industry In the 
United States, to test the curriculum and skills Identified by 
providing training In two pilot sites, and to disseminate the 
curriculum. Post secondary semiconductor manufacturing technology (SMT) 
training was offered at two community colleges-Texas State Technical 
College (TSTC), which Is also In Waco, and Boise State University 
(6SU), In Boise, Idaho. 

The project was administered by CORD, Its partner, SEMATECH, a 
consortium of 14 companies Involved In semiconductor manufacturing, 
helped CORD to provide training by defining the knowledge and skills 
needed for those trained, as well as helping them to develop the 
curricula needed. The curricula was piloted at TSTC for the general 
student population, and at BSU for retraining of Micron Corporation 
employees. SEMATECH's original role was as a link between CORD and 
Industry to locate a retraining site. 

About six months after the grant began, the director of SEMATECH 
died. With the presence of a new SEMATECH director, there was a change 
In priorities for SEMATECH. This slowed the process of locating a 
retraining site, but worked to CORD'S advantage because CORD was then 
able to work directly with industry. Before this, SEMATECH was very 
proprietary about Its Industry contacts. 
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Sixteen Micron employees completed training at BSU, and 24 
students were enrolled in the SMT program at TSTC, either as full-time 
majors or students taking SMT courses but who are actually enrolled in 
other degree programs at the college. 

Students at BSU underwent an arduous schedule, attending classes 
after work from 5:00-8:30 p.m., Monday through Thursday, for five 
months. They received 283 hours of training. Upon completion of the 
program, those trained received a SMT Pilot Program Certificate of 
Completion from BSU and program completion was noted on their internal 
Micron transcript. 

Training at TSTC included 2,400 contact hours of classroom and on- 
the-job-training. Students in the program began training in September 
of 1990 and will graduate in May 1992. The on-the-job~training 
component consisted of constructing a clean room for manufacturing 
semiconductors. Originally the program intended to manufacture diode 
packs and interpacks, however building a clean room sterile enough to 
manufacture these was unrealistic. The SMT director, who was hired 
after the grant was obtained, realized this and saw that the program 
would die without a viable on-the-job-training component. He discussed 
the dilemma with other industry professionals, who suggested 
manufacturing solar cells, which are used in calculators. The 
manufacturing process would thus remain the same as was originally 
proposed. Students were instrumental ly involved in construction of the 
clean room, an unintended but valuable benefit for semiconductor 
industry workers who frequently deal with machine breakdowns. 

The SMT Program began as a direct result of the Cooperative 
Demonstration grant. The relationship with SEMATECH was a new one, as 
was the relationship with BSU, while CORD and TSTC have had a 
longstanding and integral relationship. Several key people were hired 
to administer the program and develop the curriculum. 
The third-party evaluation of the project revealed: 

■ The actual partnership and program 
development model is exemplary, 
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particularly for use in high technology 
areas; 

■ The schedule for BSD training was 
unrealistic. Students were exhausted and 
expectations of them were too high; and 

■ The recruitment plan at TSTI needs 
attention* The strategy is not well 
planned. A real targeted recruitment 
plan is needed. Some television 
advertising reached a few students, but 
most recruitment was done informally and 
from pirating other TSTI programs. 



The evaluator also reported that all 12 tasks as outlined in the 
proposal were met to varying degrees, and CORD is in the process of 
responding to these recommendations to ensure the future success of 
this continued program. 

Dissemination was one of the tasks proposed, and It has taken 
place on a variety of levels. National dissemination of the SMT model 
has taken place at the National Coalition of Advanced Technology 
Centers' (NCATC) fall and summer conferences. A conference was also 
held at tSTC in May to disseminate the model. CORD also gave 
presentations at Rensselaer Polytechnic Institute, Applied Materials 
Corporation, and published information about the model in Economic 
Development Commentary and six issues of NCATC Newsletter . The SMT 
curriculum developed by CORD will eventually be sold to any college 
that is interested. 
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Clackamas Community College 
Oregon City, Oregon 

The Cooperative Demonstration grant to Clackamas Community College 
(CCC) was used to create a model manufacturing program in precision 
metals. The program had two goals: to train entry level employees and 
to upgrade the skills of current employees in response to technological 
change. The project also targeted special groups, including unemployed 
and displaced workers, injured workers, career changers, immigrants, 
women, and other economically disadvantaged people. 

The two major partners for the project were the Oregon Precision 
Metal Fabricators Association (OPMFA) and the Northwest Screw Machine 
Products Association (NSMPA). The partnership with the OPMFA began in 
1988 when the association asked CCC to develop and conduct a training 
program specifically for the precision sheet metal industry. The 
associations agreed to loan CCC the equipment to be used in training. 
CCC then planned and offered a series of evening courses, first at the 
main campus and then at a new 6,800 square foot training facility 18 
miles south of the main campus. The Cooperative Demonstration grant 
allowed CCC to expand the curriculum, add a basic skills component, and 
target special populations of students. 

Under the grant, project staff developed three major training 
packages: a work-readiness course, a certificate program in precision 
metal fabrications, and a certificate program in screw machine 
technology. The six-week "Work Readiness" course was designed to 
provide entry level trainees with the basic skills required for 
employment in the precision manufacturing industry. As the project 
progressed, the course was renamed "Introduction to Precision 
Manufacturing Technology" and expanded to eight weeks. The course 
outline included nine areas: computation skills, basic linear measuring 
instruments, blue print reading and flat pattern development, project 
driven operation of shop machinery, industrial operation (through 
tours), communication skills, forklift operation, first aid and CPR, 
and work readiness. The forklift training was optional and added to 
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provide students with a specific skill for entry level work in many 
companies. The project paid half the cost of the $50 forklift license, 
and the student paid the other half. Because of insufficient time and 
funds, the first-aid and CPR classes were dropped but later reinstated. 

The work-readiness program was offered six times during the grant. 
A total of 79 students entered the programs, and 65 students (82 
percent) completed training. Approximately 80 percent of the students 
were placed into jobs. Twelve students chose not to seek immediate 
employment, seven entered the nine month certificate program, two 
enrolled in unrelated college programs, one student returned to high 
school, one entered the armed services, and one (a professional 
musician) chose not to seek employment in the metals industry. The 
program was offered again in spring and summer 1991 (beyond the 
requirements of the grant). These programs lasted eight weeks, and 
each of the 18 students who attended the two sessions paid a fee of 
$1,195. All students completed the course, and five were employed as 
of October 1991. 

The certificate program in precision metal fabrication and the 
certificate program in screw machine technology were both developed by 
project staff. The project hired three instructors from private 
industry to develop the curriculum and to teach the courses. The 
programs were nine months long and allowed open enrollment at the 
beginning of each term. The maximum number of students allowed in each 
program at any one time was 15. Nine students graduated in precision 
metals by the end of the grant and seven continued into the autumn of 
1991. Eight of the nine graduating students (89 percent) are employed 
(one was under 18 and too young for employment in the industry). Nine 
students graduated from the screw machine technology program by the end 
of the grant, and nine continued into the fall 1991 program. Six of 
the nine graduating students are employed at local screw machine 
companies. One student moved out of state and chose not to accept 
employment in the industry. Two other students graduated at the end of 
August 1991, one of whom started his own company. 
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The metal fabrication curriculum was assembled and published in a 
book called The Shear Edge . The text was copyrighted in 1990 by the 
Oregon Advanced Technology Consortium (OATC)» It contains 25 chapters 
and two appendices, one of which is a glossary* The screw machine 
curriculum also is available as are the competency based curricula for 
the work-readiness program* 

Students were recruited through a variety of efforts* Project 
staff visited local high schools, alternative schools, employment 
offices, adult and family service offices, community colleges, the 
Refugee Center of Oregon, local JTPA and PIC offices. Urban Leagues, 
dislocated-worker programs, and the Life and Career Options classes for 
abused women at CCC. The project received numerous referrals from 
private vocational rehabilitation and insurance groups seeking 
retraining for injured workers* The staff put ads in local newspapers, 
sent specialized mailings to individual industry association members 
and members of the Society of Manufacturing Engineers, and arranged for 
public service announcements on KUFO Radio. The programs and courses 
also were printed in the CCC term schedule of classes, which is mailed 
quarterly to 100,000 residences and businesses* As part of the 
recruitment effort, the project produced two 10-minute videos promoting 
the job opportunities in the two areas and the two certificate 
programs. The project also produced a 30-minute repeater videotape 
containing a 75-second introduction to the CATC, which consists of CCC 
and three other community colleges that provide technology transfer and 
training to Oregon industry. 

Applicants were assessed first through a standard college 
placement exam in math, reading, and writing. If the applicant passed 
the exam, he or she met with the program instructor for a personal 
interview and performance tests involving spatial relationships. The 
personal interview allowed the instructor to adapt course content to 
the needs of the immediate group. Applicants not passing the test were 
referred to refresher math courses prior to entering the program. 
Students accepted by the program were provided free training. 




E-10 

Of the 79 students enrolled in the six sessions of the work- 
readiness program, 35 percent were female and 18 percent were minority. 
Half the students were age 25 to 35. Of the 16 students in the 
Precision Metal Fabrication Program, 12 were male and four were female, 
and two were ethnic minorities. Of the 16 students in the Screw 
Machine Technology Program, 15 were male and one was female; none were 
ethnic minorities. In spite of advertisements in local newspapers 
targeted to minorities, there was relatively low participation in the 
project by minorities for two reasons: first, training was located at 
a facility 18 miles from the main CCC campus and not accessible by 
public transportation. Most of the unemployed and underemployed 
targeted population lived in Northeast Portland and did not have 
private transportation to reach the center. Second, the use of the 
math and reading college placement exam screened out any minorities not 
having the necessary background to score highly enough on the tests. 

The two partner associations played an active role in the project, 
and involved several other companies as well. The partners identified 
competencies needed in training, reviewed the curriculum, helped 
advertise the project, sent students to participate, donated funds to 
help with equipment maintenance and tooling costs, and arranged for 
short-term training for project staff. They also paid to print 10,000 
additional brochures and 150 videos for the project. One company owner 
and association member even served as a part-time instructor. The 
OPMFA arranged for U.S. Amada Limited to loan CCC a new $250,000 CNC 
laser cutting machine and an $80,000 bending machine. The equipment 
was loaned with the understanding that U.S. Amada could bring potential 
customers to CCC to see the equipment in operation. Other companies 
loaned other types of equipment as well or metal stock to be used in 
class exercises. 

Grant funds also were used to pay for staff development. 
Instructors for the work-readiness and certificate programs attended a 
"train the trainer" provgram during the summer of 1990. The three 
primary instructors all came directly from industry and had no prior 
teaching experience. They attended a "Power Presentation Skills" 
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seminar later in the year as a follow up to their earlier training. 
This seminar was designed to make instructors more effective at 
teaching adult and disadvantaged students. In addition, CCC faculty 
received training on the equipment loaned to the center. One 
instructor attended a weeklong seminar at U.S. Amada to learn the 
operation of the CNC laser cutting machine, and another support staff 
member attended for two weeks. The instructors also attended training 
in Brooklyn on the CNC plasma cutting machine. One instructor attended 
a course on the coordinate measuring machine. 

In addition to the lack of public transportation to the training 
center, the project had two other implementation problems. First, the 
downturn in the local economy after the start of the project made 
employers hesitant to hire new workers. It was difficult to place 
graduates and students had to spend more time looking for jobs than 
originally anticipated. Second, the donated machinery was expensive 
to transport to the training center and created maintenance and tooling 
needs. The original grant budget had not provided resources for the 
maintenance of equipment, and instructors had to spend time finding 
additional resources and materials. 
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I Fox Valley Technical College 

I Appleton, Wisconsin 

I Fox Valley Technical College (FVTC), located in Appleton, 

Wisconsin, offers 59 associate degree and vocational programs to more 
I than 5,000 students, as well as contracted training to approximately 

■ 17,000 local industry employees per year, Appleton is in a part of 
I Wisconsin that is highly oriented to manufacturing • 

I The goals of the Computer-Integrated Manufacturing Program (CIM) 

were to demonstrate a cooperative approach to provide training and 

I technical assistance in CIM to local manufacturers and develop a model 

that could be used by other postsecondary educational institutions. 

I The specific goals of the CIM program, as stated in the proposal, 

were to provide: (1) Orientation to CIM for small businesses through a 

I collegewide CIMulation Laboratory; (2) Cost-effective training in CIM 

■ via a quality approach to instruction; and (3) Comprehensive, yet 

m affordable, support in CIM planning and implementation from an expert, 

I faculty-student technical assistance team. 

The first steps needed to carry out these goals were to increase 
I the client base and market CIM. To that end, individual instructors 

reached out to industry. Also, the CIM Solution Demonstration (CSD), a 
I storybook demonstration on computer, referred to as the CIMulation 

laboratory in the proposal, was instrumental in promoting marketing. 

■ It was put into place with Cooperative Demonstration grant funds. 

■ The CSD shows how a small manufacturing company uses the 
g technologies of an integrated system to improve its management, 

■ operation, and responsiveness. The live demonstration tracks an 
engineering change from a request for quotation to the shop floor where 

I a prototype part will be cut to meet a customer's specifications. All 

departments--such as management, operations, and production-explain 

I their role in this engineering change, and the demonstration concluded 

with a final quotation being printed for the customer. Conversion to 

■ CIM technology for this company resulted in improved response time and 
" an improved market share. Every Thursday, businesses interested in CIM 
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were given two-hour tours through the CSD. To Increase local business 
exposure to the CIH Solution Demonstration, a student phoned local 
businesses to Invite them to the demonstration, a practice which 
continues. FVTC used grant money to supplement the cost of training as 
an Incentive to get companies to participate In CIH. CIM training 
usually was held on the campus of FVTC at the Bordlnl Center, which was 
especially built to provide corporate training and is a symbol of 
FVTC*s commitment to Industry. This arrangement was preferred to on- 
site training because there would be fewer interruptions, but in some 
cases on-site training was essential because specialized equipment was 
only available there. Once CIM was implemented on-site, a CIM 
instructor would spend time at the company making sure implementation 
was complete. 

Not only was the CIM curriculum offered to companies, it was also 
integrated into other FVTC degree programs, such as accounting, data 
processing, and printing and publishing. Approximately 1,300 students 
at FVTC completed a course that was infused with a CIM curriculum 
component, and 216 persons already employed by business and industry 
completed training. 

Prior to the grant. Fox Valley entered into a relationship with 
IBM, its. partner, because the college was chosen by IBM to become a 
member of the CIM Alliance. The CIM Alliance is an alliance of 70 
colleges and universities throughout the United States that are active 
in CIM technology. The CIM Alliance allowed Fox Valley to exchange 
technology and develop CIM curriculum. IBM donated computers and 
computer programs. 

There was a small change in strategy from what was proposed in the 
application to what actually happened with CIM training. One goal of 
the project was to put together a team of instructors to go out into 
industry and offer a CIM system that affected the whole company. This 
has not worked well, however, because area companies would rather 
undergo piecemeal change rather than company-wide change. Therefore, 
CIM training more typically takes place using one particular 
application, such as CAD to CAM, using AutoCAD software. By taking on 
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these small CIM application training sessions, CIM instructors hope to 
spread the word about CIM availability and the possibility of company- 
wide change. 

The third-party evaluation was particularly valuable to the CIM 
staff, causing them to institute some changes that worked to the 
benefit of the program. The strengths of the FVTC program identified 
by the evaluator included: (1) FVTC is truly operating a CIM system. 
The evaluator had seen many CIM systems, but this is the first one that 
really combines all major components of CIM, such as managerial, 
technical and manufacturing, and engineering; (2) Institutional and 
local support for the program were great; and (3) Interest and 
abilities of participating staff were great. 

One recommendation that the evaluator made during the first 
evaluation site visit was that communication among project staff 
members be improved. There needed to be more of an understanding of 
all of the components of CIM. 

Communication had improved by the time of the evaluator* s second 
visit. For instance, business instructors were using shop floor 
language and vice versa. 

Dissemination has taken place on a variety of levels. Videos were 
produced with grant funds that explain CIM in several different ways. 
One video is of the overall CIM program, and all subsequent videos 
explain CIM*s application in business, design, publishing, and 
manufacturing. Presentations about the CIM program were also made at a 
Leadership 2000 seminar and a Society of Manufacturing Engineers 
conference in Chicago. Also, a CIM Breakfast, at which one of the CIM 
videos was shown, was held for 100 local business people, politicians, 
and the press. 
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Hampden County Employment and Training Consortium 
Springfield, Massachusetts 

The Hampden County Employment and Training Consortium applied for 
a Cooperative Demonstration Program grant In FY 1989 after having 
received Its first Cooperative Demonstration grant the year before^ 
The first grant supported "Project CREATE" (Cooperative Resources to 
Enhance Access to jobs through Technology Education), which provided 
167 adults with hands-on training in the maintenance and repair of 
computerized numeric control (CNC) machinery, high tech automotive 
repair, and printing and graphics. The first grant offered eight skill 
training programs and 1,133 hours of training across all eight 
programs. The second grant supported "Project: High Tech *90,'' a 
program designed to recruit and train disadvantaged youth, women, 
minorities, and underskilled adults in approximately the same three 
career fields: automobile repair, graphics and printing, and machining. 
The first grant ran from January 1989 through June 1990, and the second 
grant ran from July 1990 through December 1991. 

Much of the project Infrastructure and partnership arrangements 
from the first grant were carried over to the second grant. The same 
consortium staff were used on the project, although staff shifted 
positions because of the sequencing of projects. The administrative 
offices were the same, as were four of the partner organizations: 
Springfield Technical Community College, Dean Vocational Technical High 
School, Westfield Vocational Technical High School, and the 
Massachusetts Career Development Institute (MODI). The course content 
and structure also were similar from one grant to the next, and all but 
one instructor was rehired. 

There was, however, an Important difference between the two 
grants. "High Tech *90" emphasized much more heavily the recruitment 
and training of disadvantaged and nontraditional populations. Both 
project staff and partner staff were Involved in recruiting students 
for the no-cost, open enrollment programs. Staff developed flyers that 
were sent to community-based organizations, prepared a videotape that 
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was played at local social services office, set up offices at the 
partner organizations, and ran cable TV and radio announcements. There 
was no screening for either prior skill levels or motivation (a fact 
later regretted), and all applicants were accepted. Project staff 
worked with the automotive instructor at Dean to translate training 
materials into Spanish for Hispanic students. Dean also hired a 
bilingual instructional aide to work with the classes. Counseling 
staff at several of the schools tracked the High Tech '90 students more 
closely than regular students. 

The project set recruitment targets of 25 percent Hispanic and 40 
percent female. The training programs were able to enroll a higher 
percentage of minorities and females than the regular programs offered 
by the partners, and several schools are continuing their minority 
recruitment after the end of the grant. The project, however, did not 
meet its target. The project was able to achieve 19 percent Hispanic, 
16 percent black, and three percent Indian participation. 
Approximately 21 percent were female. One of the most difficult 
aspects of minority recruitment was getting some staff members in the 
partner organizations to change their attitude about women in 
nontraditional occupations and the involvement of minority and other 
disadvantaged populations. The partners were accustomed to being paid 
by the JTPA agency based on student performance and so were reluctant 
to take students who might not finish training or who were ethnically 
different from their other students. 

The project offered seven training programs in six different areas 
(the pretraining program was repeated twice). The first was a training 
program for 125 secondary students from nontraditional backgrounds to 
familiarize them with technology career fields. This program was 
offered twice by Springfield Technical College, and each program 
consisted of ten hours of training. Of the 125 students starting, 115 
completed the training. Because these were secondary students still 
enrolled in school, none were placed in jobs. 

The second program was 400 hours of training in desktop publishing 
and printing at MCDI. Of the 20 students participating, 13 completed 
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training, and six were placed in jobs. MCDI hired a new instructor 
from industry to teach the course because the instructor from the first 
grant was now teaching regular classes. The start of the training was 
delayed three months while MCDI searched for a new instructor. The new 
instructor then had one month to develop his course using pieces of 
courses and materials from similar courses in the area. One of the 
project staff served as the instructor for the desktop publishing 
component. The class applied its training to the publishing of a 
project newsletter. Access , which was printed on donated paper and 
mailed to approximately 100 businesses and organizations in the area. 

The third program was 254 hours of basic machining training at 
MCDI. Of the 20 students participating, 12 completed their training, 
and six were placed in jobs. One of the female students trained under 
the first grant became an instructor for the second grant. 

The fourth program was 254 hours of automotive repair training at 
Dean. Of the 24 students participating, 16 completed training, and 
eight were placed in jobs. 

The fifth program was 96 hours of computer aided design/computer 
aided manufacturing (CAD/CAM) training at Springfield Technical 
College. Of the 22 students participating, 18 completed the course. 
All the students were employed at the time of training, and so there 
were no further job placement activities. 

The sixth program was 48 hours training on computerized numeric 
control machining for currently employed machine shop workers. 
Training was done by Westfield Vocational Technical High School. Of 
the 15 machinists trained, all completed training and were working at 
the time of graduation. 

During the duration of the project, the economy of western 
Massachusetts was severely affected by the recession, and employment 
opportunities for new entrants into the automotive and machining fields 
became scarce. Employment opportunities in printing and graphics arts 
were somewhat better but still reduced from pre-recession levels. 
Consequently, the project had very low employment success: 25 percent 
entry for automotive, 40 percent in machining, and 55 percent in 
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graphic arts and printing. The overall impact for those employed was 
lower starting wages than had been expected, longer job searches, and 
still uncertain futures. As the recession worsened, project staff 
shifted emphasis from skill training to support services such as resuae 
writing, interviewing skills, and job placement. 

There was no advisory board for the project and no general 
meetings of representatives from the partner organizations. The 
project staff's experience in the first grant showed them that 
attendance at such meetings was sporadic and that it was more efficient 
to interact with the partner organizations one-on-one. In addition, 
the consortium was also a research organization that collected job 
market information on a regular basis; no additional information was 
needed from private industry. The staff time that would have been 
spent on advisory board activities was directed toward starting 
discussion groups with high school students regarding their career 
expectations. 

The curriculum packages developed by the project (and its 
predecessor grant) are provided free to anyone requesting copies. The 
packages are suitable for use by other vocational schools and technical 
colleges. Procedures used for recruiting students may be of use to 
other schools, as evidenced by one partner school having expanded its 
regular recruiting process to incorporate the project's procedures. 

The project used the newsletter to publicize the content and 
accomplishments of the project. Although the newsletter was sent to 
over 600 individuals and organizations, including the other grantees, 
there were few requests for information. Two other grantees came to 
review the project, but it is not known if any of the project's 
features were implemented elsewhere. Copies of the CNC curriculum were 
mailed to anyone requesting them. Project staff also made presenta- 
tions at state and national conferences, e.g., AVA. 
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Home Builders' Institute 
Washington, D.C. 

The grant to Home Builders' Institute-a subsidiary of the National 
Association of Home Builders (NAHB)-was designed to develop a model 
curriculum to train and certify employed master electricians and 
second-year apprentices as installers of SMART HOUSE. SMART HOUSE is a 
high-technology home automation energy, distribution, and control 
system, currently under development by an NAHB spin-off organization. 
The proposal anticipated instructional design and development, 
production of training materials, student recruitment, instructor 
training, pilot training at three sites, assessment, and dissemination 
to participating vocational institutions. 

As executed, the project encountered problems that led to changes 
in its scope and timing. A delay in the development of the SMART HOUSE 
technology and failure to reach agreement with SMART HOUSE developers 
on the nature of installer training led to a shift in curriculum 
development. Instead of SMART HOUSE, the curriculum that was 
ultimately developed instructs students on the installation of a 
generic home automation system that uses existing home wiring systems. 
As a result of the SMART HOUSE problems and subsequent changes, 
curriculum development was set back between 12 and 18 months. In 
addition, there was a change in project directors after the first year, 
and several changes in staff writing the instructional materials. 

Once the decision was made to develop the more generic home 
automation curriculum, information was sought from a variety of 
sources. An advisory committee was formed with members representing a 
variety of home automation manufacturers as well as users. Committee 
members reviewed curriculum drafts. Most of the curriculum writing was 
carried out by staff and consultants of Home Builders' Institute. A 
final, revised curriculum for an 18-hour installer- training course was 
slated for completion in December 1991, but the project has applied for 
f,n extension to June 1992. 
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Three sites were recruited for pilot testing, which has taken 
place at two of them. The project has trained 15 instructors and 16 
students at Mid-Florida Technical Institute, and 1 instructor and 10 
students at Tidewater Community College. Students included working 
electricians as well as current students at the two schools. The 
training at Mid-Florida was conducted by a Home Builders* consultant, 
while the training at Tidewater was conducted by one of the trained 
trainers. Pilot testing at the third site was scheduled for December 
1991, but canceled because of Insufficient registration. It may be 
held in March 1992. In addition, some Instructors from pilot sites 
have received SMART HOUSE training in order to fulfill agreements made 
with the sites prior to the shift in project emphasis. 

As a result of pilot testing, the curriculum is being revised. 
Student evaluations suggested that training be made less elementary and 
more technical. In addition to changes in the course text, the project 
is developing audio/visual support (videos and overheads to introduce 
the Home Automation concept) as well as templates for laboratory hands- 
on manipulation. 

Clearly, dissemination of the curriculum has not yet occurred, as 
the curriculum is not completed. Dissemination was originally slated 
to be conducted by Partners for American Vocational Education as well 
as Home Builders* Institute. 
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Howard Community College 
Columbia, Maryland 

The grant to Howard Community College was designed to develop a 
model curriculum for a refresher course for persons returning to 
nursing after an absence of five or more years. The proposal 
anticipated changing an existing refresher course by updating the 
competencies taught and by developing and using interactive videodisc 
technology (rather than having to purchase expensive high-tech 
equipment). Howard Community College planned to work on this project 
in combination with Essex Community College and two local hospitals 
that provide clinical sites for refresher course participants. 
Although the original proposal did not go into detail, the project 
envisioned a major revision in the refresher course already offered, 
with videodiscs developed under the grant playing a major role in 
instruction. As the proposal stated, 

A 12-week, 150-hours course will be held in the 
winter/spring semester of 1991. Eighty hours will 
be spent using the videodisc, attending class 
lectures, and in lab practice; 65 clinical hours 
will be devoted to patient/client contact, with 
another five hours in conference in participating 
hospitals. The classes will be led by the same 
instructor who had previously taught these courses 
so that significant differences in class outcomes 
can be attributed primarily to the addition of the 
videodisc rather than a change in instructors. 

As originally planned, 40 students would be recruited for the course, 
and the intended goal was a 90 percent completion rate. In addition, 
the curriculum and discs would also be "piloted" by nursing staff and 
students at a minimum of 10 institutions. 

As executed, the grant led to the development of a seven-minute 
recruitment film and three interactive videodisc "sides" (like the side 
of a tape or record) on the subject of volumetric pumps. The content 
and scripts were developed by project personnel (director and 
instructor) and reviewed by an advisory committee that included 
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hospital representatives. Actors and project personnel were used to 
tape the videos. The videodiscs, along with a workbook, were 
subcontracted to SETS, a marketing firm, and Digital Video Corporation, 
which actually programmed the discs. Project staff and the advisory 
committee reviewed the discs di-ring development, and one of the 
interactive videodiscs was also reviewed by nursing staff at seven 
other teaching institutions. Programming for the videodiscs was still 
being refined in December 1991, six months after the grant ended^ 

The videodiscs were incorporated into the refresher course for 
nurses, but delays in the development of the discs meant that students 
were exposed to a work in progress, rather than to a finished set of 
discs. Twenty-nine students were recruited to the refresher course. 
As noted by the third-party evaluator, other than the introduction of 
the videodisc, few changes were made in the refresher course as the 
competencies taught in the course were "found to be current" after 
being reviewed by project staff and the advisory committee. Estimated 
instructional time devoted to discs was about six hours. The evaluator 
found that students who used the interactive discs in instruction 
appeared to be less anxious about taking care of patients on machines, 
compared to students who took a similar course without the discs. 

One videodisc was sent to 13 institutions and seven of those 
institutions completed the feedback forms from students and faculty 
after using the videodisc. According to the third-party evaluator, "A 
total of 115 students and 12 faculty reviewed side one of videodisc 
one..." and that number includes the 29 students at Howard and Essex. 

Twenty-nine students were recruited to the refresher course at 
Howard and Essex, and all but one completed the course. Not all of the 
students sought work as nurses, and because of staffing cutbacks, 
neither hospital participating in the project hired anyone who had 
completed the refresher course. One hospital had major personnel and 
organizational changes that interfered with its involvement. At the 
time of the foUowup, 15 of the 29 participants were employed and 10 
were in settings where high-tech equipment was being used (described as 
"nursing related work"). 
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High-technology aspects of this project included the use of 
interactive videodiscs as well as training to work on high-tech 
equipment (computer-driven volumetric pumps)* The project was a new 
one for Howard Community College, which had never developed an 
interactive videodisc* The nursing program was well established 
however* The college had previously received a grant from the state to 
develop the refresher course* No new staff were hired for this 
project; all staff were already employed at Howard, Essex, or the two 
hospital s* 

Dissemination was delayed because programming for the interactive 
videodiscs needed further revision from the feedback from the pilot 
sites. The project director has presented information on the discs at 
several national meetings. She also credits the disc development with 
having received a Fulbright Scholarship for next year. Howard will 
sell the discs in conjunction with SETS* The project director 
indicates that the Federal monitors have okayed an arrangement in which 
SETS and Howard will share the profits, with Howard receiving 20 
percent. 



ERIC 



253 



I 
I 

I 
I 
I 



E-24 

Illinois Eastern Community College 
Olney, Illinois 

Olney Central College (OCC) is one of four branches of the 
Illinois Eastern Community College (lECC) system* OCC is a small 
college offering two-year associate degrees in a predominantly 
agricultural area of southeastern Illinois* The training in 
semiconductor technology program (Semi Top) began as a result of the 
Cooperative Demonstration grant* The purpose of the project, as stated 
in the proposal, was to train students to become technicians and 
operators in the semiconductor or chip fabrication industry and to 
establish this program as a model of business/college partnerships in 
vocational education. The objectives were to: 

■ integrate additional technical course 
offerings at lECC into the semiconductor 
technology, level I program, which will 
be offered as part of the lECC 
curriculum; 

■ recruit at least 20 students into the 
program by August 1990; 

■ train these students in a program that 
will lead to industrial employment or 
training at Level II; and 

■ develop the Level II program by June 30, 
199K 

This project intended to train students in the classroom and 
introductory laboratory experience (Level I) and develop an advanced 
laboratory curriculum (Level II)- The second year of the program was 
to take place in a simulated microchip laboratory or "clean room" on 
01ney*s campus, but this never happened. In the application, a diagram 
of the sterile laboratory was provided. According to the project 
director, there was an error in the type of simulated laboratory 
environment discussed in the proposal-only an authentic sterile 
environment would make an adequate laboratory. The evaluator 
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eventually discovered that there was miscommunlcation between the 
project director and OCC about exactly what classroom space and 
laboratory the college would provide for the SemiTop project. The 
project director spoke of "leveraging" one grant with another and 
sought assistance from the State of Illinois to fund the sterile 
laboratory. Jhis never occurred, and no other plans currently are 
under way to secure funds. The result was that the 19 students 
originally enrolled in SemiTop training did not receive a two-year 
degree in Semiconductor Manufacturing as originally planned, however 
they did complete the Level I courses. 

One student who did complete the class got a job with Intel 
Corporation. Eight students continued their education at OCC in a 
different program, and seven students did not complete Level I 
training. 

Level I training was taught by instructors already employed at 
OCC. A lab assistant, who also served as a full-time tutor, was hired 
as a result of the grant. Students typically took four or five 
courses-a full-time course load for the first and second semesters. 

This partnership was unique because Olney Central College's 
partner, Intel Corporation, was located hundreds of miles away in 
Phoenix, Arizona. Intel's major role was to supply equipment and 
contribute to staff development and curriculum development during four 
meetings held at OCC during the summer of 1990. Instructors received 
training which enabled them to include a semiconductor component to 
their already-existing curricula, in such areas as math, physics, 
computers, and chemistry. Intel also was going to install the 
equipment it donated as part of its role in the partnership, but this 
never came to pass. 

SemiTop dissemination included the project director's attendance 
at a SEMATECH (a consortium of 14 semiconductor manufacturing 
companies) meeting, conferences, and distribution of the curricula to 
anyone requesting it. A paper was written about the SemiTop experience 
and was delivered to the National Association for Science, Technology, 
and Society in Washington, D.C. in February 1991 at the Technological 
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Literacy Conference. A SemiTop manual was also disseminated to the 
ERIC (Educational Resources Information Center) Clearinghouse. 
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Indian Hills Community College 
Ottumwa, Iowa 

The Indian Hills Community College's (IHCC) Demonstration Model 
Project was developed to provide a degree-bearing second year course of 
vocational study to non-traditional students seeking high technology 
training. The program was developed as a flexible competency-based 
program offering an opportunity for skill enhancement and a college 
degree. As stated in the proposal, the project offered three paths of 
study: 

■ a course of study in Personal Computing 
resulting in a diploma or an associate 
degree; 

I an Associate of General Studies (AGS) 
degree in Advanced Technology, 
specifically tailored to meet student and 
industry need, and; 

I courses to be taken for skills upgrading 
and enhancement. 

The program was designed as an individualized self -paced approach to 
study, utilizing flexible study time, lab time, industry 
representatives, educational facilitators, and other college personnel. 
The project offers the non-traditional student, including women, 
minorities, displaced homemakers, and others seeking advancement 
opportunities, the chance to design a degree program made up of a 
combination of interdisciplinary courses technically- focused to meet 
real world needs. 

The project featured three educational components each being 
twelve months in length. All the programs were offered in the evening, 
and were self-paced with flexible hours and facilitators present. 

The Microcomputer Specialist Diploma component was a 24 semester 
hour offering in Personal Computing. It was a one year program for 
students seeking proficiency in microcomputer software applications and 
resulted in a diploma. This component when combined with an 

26,? 
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I Electronics/Computer Operations diploma from IHCC's traditional 

■ offerings could result in an Associate of General Studies (AGS) in 
I Advanced Technology degree. 

I The AGS in Advanced Technology component was designed as a second 

year degree-bearing program in high technology. The first year of 

I training was obtained by students that completed the college's 

Electronic/Computer Occupations diploma program through the traditional 

I curriculum prior to starting the AGS training. 

Utilizing the AGS program, with assistance from an academic 

I advisor and an advisor from the area industry, students combined High 

■ Technology courses and credits from other disciplines, to gear skills 

■ specifically to suit the needs of area industry. However, a student's 
I AGS in Applied Technology plan was subject to IHCC Academic Standards 

Committee approval prior to the start of course work. 
I The third component was made available to train students in second 

year High Technology programs on a course-by-course basis while 
I accumulating college credit. The courses allowed current employees of 

industry to upgrade skills or train for new responsibilities that would 

■ enhance mobility and productivity on their jobs. 

" In all three components, industry volunteer advisors were enlisted 

B on an as needed basis to assist in helping students customize their AGS 

I programs. Also eight current IHCC instructors/specialists were 

utilized as "experts" to assist students in understanding high tech 
I specialty concepts in each component. These experts were used four 

hours per week each to supplement the full-time educational 
I facilitators during the non-traditional delivery hours. 

In this project, students' time is flexible allowing them to put 

■ in their contact hours at convenient times based on work schedules or 

■ child care, etc. The students must, however, start and complete their 
M required contact hours in accordance with the college term calendar. 
m It requires good time management on the part of the student and 

consistent motivation and monitoring on the part of the facilitator. 
I As specified in the proposal, the program goals were: (1) to 

provide outreach efforts to 200 potential program participants; (2) 
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register 25 and graduate 15 students in the Microcomputer Specialist 
Diploma component; (3) register 15 and graduate 12 students in the AGS 
in the Advanced Technology component; (4) Register 20 and have 20 
students complete at least one course in the course-by-course 
component, and; (5) of the total number of participants, at least 66 
percent should be women* 

The actual numbers registered were greater than expected in some 
components and less than others. However, the overall total 
registrants exceeded the original goals. Of those registered in the 
Microcomputer Specialist Diploma component, five completed their course 
work. The AGS in Advanced Technology component graduated 10 students, 
and the Course-By-Course component had 14 completers. 
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LTV Steel Company 
Whiting, Indiana 

The LTV Steel Company applied for a Cooperative Demonstration 
Program grant to support a portion of its apprenticeship program in 
Electronic and Instrument Repair. The apprenticeship program was 
started in 1988 by the Employee Development Department as part of LTV's 
$320 million in capital investments in its East Chicago plant. The 
installation and modification of new equipment required the creation of 
a new occupation-Electronic Instrument Repair Technician (EIRT)-and the 
retraining of a portion of the existing workforce. An EIRT 
repairs/replaces temperature, pressure, and flow measurement instrument 
components as well as the digital and analog electrical systems with 
which they interface. 

The EIRT apprenticeship training is a four year program providing 
8,320 hours of classroom and lab instruction. Trainees are given one 
day of classes a week at the Calumet Campus of Purdue University, one 
day per week of lab at the training facility at LTV Steel, and the 
balance of time in on-the-job training. Two courses are taught each 
semester, and a semester lasts 14 weeks. The four-year program is 
considered a more desirable arrangement than having people with a full- 
time two-year associate degree plus two years on-the-job-training. 

The EIRT training is open to all of the plant's hourly employees. 
Interested workers apply to the program and must pass a reading 
comprehension and math assessment to be eligible. Apprenticeship 
positions are then awarded to eligible workers on the basis of plant 
seniority and consistent with "Consent Decree No. 1" signed in the U.S. 
District Court for the Northern District of Alabama in 1974. The 
decree specifies minority representation in apprenticeship programs in 
selected craft families. Once a worker has started in the EIRT 
program, he or she cannot withdraw for a minimum of 45 days. 
Apprentices receive wages stipulated in the current labor agreement and 
are expected to earn over $101,500 for the 8,320 hours of training. 
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The first cohort of apprentices started in June 1988 and consisted of 
25 students; 17 students still remain in the program. The second 
cohort started in September 1988 and consisted of 25 students; 16 
students remain in the program. 

LTV's grant was used to support 18 of the 48-month EIRT training 
for the third cohort of apprentices. The third cohort started in 
January 1989, and 28 students completed the grant- funded portion. The 
LTV Manager of Employee Development Programs applied for the grant 
(LTV's first Federal grant application) after hearing about the 
Cooperative Demonstration Program from an official in the Illinois 
Department of Labor. The Manager had contacted the Department in an 
effort to find public funding for equipment needed to establish a 
process control lab at the Calumet Campus. LTV had received small 
grants from the state in the past to pay for various job training 
activities and LTV hoped to find similar funding. LTV agreed to donate 
approximately $150,000 of equipment for a lab if Purdue would handle 
the installation and software programming. 

The 18-month component funded by the grant consisted of three 
sections of training: 

(1) Orientation (11 weeks) 

General Program Orientation and Shop Location; 
General Shop Safety 
Basic Mathematics 
Principles of Basic Physics 
Basic DC Electricity 

(2) Data Collection and Communication Devices and Systems 
(37 weeks) 

Safety 

Electronic Circuits I 
Hydraul ics/Pneumatics 
Print Reading 
Basic Electrical AC 
Electrical /Instrument Devices 
Troubleshooting Techniques 



(3) 



Control Devices and Systems (30 weeks) 
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Safety 

Electronic Circuits II 

Digital Devices and Systems 

Instrumentation 

Basic Computing and Hardware 

Instrument Test Equipment and Analyzers 

Troubleshooting Techniques 

The curricula for these courses were developed prior to the start of 
the grant by the private partner, the Electrical Engineering Technology 
Department at Purdue University-Calumet. The department assigned an 
instructor to work with LTV when LTV solicited the proposal for the 
EIRT training program in 1988. LTV had solicited similar proposals 
from other local vocational and educational institutions, but Purdue 
was the only one to respond with interest in developing curricula and 
lab experiments specific to LTV's needs. LTV selected Purdue in late 
spring of 1988 in spite of the much higher cost of their proposal. 

The Purdue instructor visited the mill several times while 
developing the initial curriculum to observe the specific control 
systems the technicians would service and maintain. Training on the 
specific equipment was critical. Purdue could train in basics of AC 
and DC circuits and equipment and fundamentals of hydraulics and 
pneumatics but didn't have laboratory facilities for process control. 
Portions of the curriculum were already available from earlier 
apprenticeship programs at LTV, but the instructor had to conduct 
extensive rewrites to update the available technology. The plant's 
Joint Apprenticeship committee (comprised of LTV Steel management 
representatives and representatives of local 1011 of the United Steel 
Workers of America) reviewed the projected summary of work processes 
and applied for Certification and Registration through the U.S. 
Department of Labor, Bureau of Apprenticeship and Training. The 
program, was certified in July of 1988. 

Students are given Knowledge Questionnaires (KQs) and 
Representative On-the-job Assignments (RPAs) to test their practical 
job knowledge. These tests and assignments are administered through 
the Electronics and Instrument Control Department and monitored by the 
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joint Apprenticeship Committee. At the time of the grant application 
there were approximately 56 KQs and 94 RPAs developed. The Purdue 
instructor often rewrites the old exams as apprentices progress into 
new units to reflect what the instructor feels is most important and 
the latest changes in technology. 

The problems encountered by LTV in implementing the EIRT 
apprenticeship program were the reverse of those encountered by 
educational institutions receiving Cooperative Demonstration grants. 
The vocational and educational institutions LTV approached for 
curriculum development and instruction were unresponsive. Although the 
Dean of the department at Purdue was supportive of the partnership with 
LTV, other department faculty were not enthusiastic about teaching non- 
credit courses. The instructor who did take on the LTV assignment 
(with great enthusiasm and dedication) encountered problems in 
receiving tenure from the department because of his teaching in private 
industry. Purdue hired a new instructor to teach the third EIRT 
cohort, but LTV found the instructor unacceptable and negotiated with 
the Dean to free up additional time for the original instructor to 
handle the third cohort as well. The instructor has incorporated 
portions of the EIRT training into his regular university classt^. 
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Luzerne County Community College 
Nanticoke, Pennsylvania 

Luzerne County Community College's (LCCC) Advanced Technology 
Center (ATC) Computerized Numeric Control (CNC) Cooperative 
Demonstration Training Program was developed partly as the result of a 
meeting on CNC training convened by Congressman Paul Kanjorski at the 
Wilkes-Barre Pennsylvania Chamber of Commerce. The meeting was 
attended by five regional manufacturers who said they were unable to 
operate at full capacity because of a shortage of CNC operators. 
Adding support to the need for CNC operators was a national study 
conducted by the Hudson Institute, which stated that by the year 2000 
there would be a need for 61,000 "precision production workers" in the 
United States (294 of these positions would be in the area served by 
LCCC) . 

The project called for refining the existing CNC prograi, ^y using 
an interactive video CNC controller simulation in conjunction with the 
College's Institute for Development of Educational Activities (IDEA), 
making it suitable for those traditionally underrepresented in skilled, 
high technology operations. The revised program would target groups 
such as women, the disabled, and racial and ethnic minorities. IDEA 
would provide the client population with literacy and basic skills 
training, counseling, placement testing, and other academic support 
services required to succeed in the CNC program. The CNC training 
would be offered to qualifying students at no charge and include a $25 
per week stipend to help students with travel and lunch expenses. 
Relationships with the Luzerne County ^. ^stance Office, the Luzerne 
County Human Resources Development Department, the local office, JTPA, 
and regional school districts were established to build a client 
referral system. 

Private sector involvement included four companies, one labor 
council, three development agencies, and three human resource agencies, 
all of which originally agreed to participate in the planning and 
operation of the project. In addition, each participating industry in 
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its letter of support and participation, indicated its strong interest 
in providing some on-the-job-training (OJT) for program participants 
and possible employment for students who completed the program. Due to 
the downturn in the economy, companies were not able to follow through 
on the previously arranged commitment, and the program could not place 
many of the students in OJT or with local industry employers. The 
major emphasis of the program therefore was focused primarily on 
student training. 

Because computerized numerical controllers (CNCs) and the 
equipment they operate are so expensive, it is not feasible for most 
training programs to have a number of different systems available at a 
training site. To address this issue, LCCC developed and produced two 
interactive training simulation software programs, each providing one 
hour of instruction. The interactive software programs utilize 
computer-based interactive videodisc (IVO) instruction and a 
touchscreen monitor to simulate two different CNCs used by local 
companies. With this software, the user is able to press the actual 
video images of the CNC keys (displayed on the screen) and perform 
basic simulated functions. Because the two CNCs selected for this 
project are in common usage, other training institutions and job sites 
can use the materials LCCC developed. 

Originally, the software was to be developed in the first half of 
the project and used for training during the second half. However, the 
development of the interactive video took far more time than 
anticipated. The first disc was finished late, and the second was not 
finished by the end of the grant, so students were unable to fully 
utilize the IVD instruction. 

LCCC enrolled 58 students into the program. Twenty-six (44.8 
percent) successfully completed the program, and 32 (55.2 percent) 
dropped out. All of the students who completed the program were 
referred to employers and 10 were employed (eight of whom were hired by 
partner organizations). Six students continued with their education. 
The project modified existing courses in CNC, CIH, and math to 
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meet the lower ability level of students seeking non-traditional career 
fields. The project relied upon LCCC developmental study courses for 
remediation where necessary and added internship component for all 
students. The project used existing LCCC staff except for a project 
secretary who was hired just for the grant. 

The project staff exhibited the interactive video program at 
"Pennsylvania Technology *9V where they tried, unsuccessfully, to sell 
copies of the videodiscs. No decision had been made at time of site 
visit for further duplication and distribution of the discs. The 
project obtained newspaper coverage for the implementation of the 
interactive video. Representatives from local companies using CNC 
equipment, e.g., Midway Tools, have come to see the interactive video 
training for possible use in their company. Copies of the curriculum 
packages were sent to ERIC. 
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Nebraska Department of Labor 
Lincoln, Nebraska 



Job Training of Greater Nebraska (OTGN) is an agency of the 
Nebraska Department of Labor that operates employment and training 
programs under the Job Training Partnership Act. The Rural Allied 
Medical Business Occupations Program (RAMBO), funded by the Cooperative 
Demonstration grant, trained respiratory therapists, drug and alcohol 
counselors, and licensed practical nurses (LPNs). Also one student was 
trained in histology and one as a lab assistant. 

The major goals of this program were to establish partnerships, 
increase access to health care training, and create new or improved 
training. But the program's ultimate focus became one of finding jobs 
for needy people. The five objectives, as outlined in the proposal, 
were to: 



■ help solve rural Nebraska hospitals 
Allied Medical shortages based upon 
needs; 

■ provide economically disadvantaged 
individuals... with high-tech medical 
training which will lead to professional 
standing in rural Nebraska communities; 

I furnish Job Service, Department of Social 
Services, Department of Education, and 
Job Training of Greater Nebraska with the 
opportunity to refer disadvantaged 
individuals for training in high-tech 
occupations. 

I provide individual quality training in a 
high-tech field over a 12- to 18-month 
period. 

I provide on-the-job training/classroom 
training opportunities in high-tech 
medical disciplines at Saint Francis 
Medical Center and Central Community 
College in Grand Island. 
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JTGN, in Lincoln, Nebraska, teamed with Saint Francis Medical 
Center and Central Community College, both in Grand Island, Nebraska. 
The JTGN role was to recruit students and coordinate the partnership. 
Saint Francis' role was to provide on-the-job-training, and Central 
Community College's role was to provide classroom training. The 
partnership planning began one year prior to the grant, but the 
partnership itself began with the grant. 

This partnership was distinct in that it consisted of a very 
tight-knit relationship among three dedicated partners. The director 
of RAMBO at JTGN, Central Community College's institutional advancement 
director, and Saint Francis Medical Center's director of human 
services, were dedicated to RAMBO and put forth many hours on their own 
time to ensure the students success. 

Support services were also a unique aspect of this program. 
Remedial help in the form of supplemental coursework taught at CCC was 
provided to students who needed it. Financial support was also given 
to students, many of whom were on welfare, to pay for tuition, to help 
them relocate to Grand Island, and to purchase necessities such as 
eyeglasses and suitable clothes for job interviews. 

All training programs were one year, and all included on-the-job 
and classroom training. The LPN program included 500 clinical hours. 
Drug and alcohol training included a 30-day in-patient therapy 
component; the clinical component included observing group therapy 
sessions, conducting counseling sessions, and learning how to write 
master treatment plans. Trainees also were given a caseload for 
several months at the end of their training, under the observation of 
program staff. 

Some problems with RAMBO included student retention. Twelve 
students in the program dropped out for a variety of reasons, ranging 
from academic difficulty to problems with self-esteem. RAMBO staff 
continually urged these students to stay in the program. Eventually 26 
students finished the RAMBO program, all hut two of whom were hired 
following graduation. Some students also had problems regarding State 
certification requirements for drug and alcohol counselor and certified 
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respiratory therapist licensing, although petitions of waiver for the 
State requirements were submitted and granted. 

The partners are trying to secure State funding with the help of 
the lieutenant governor in order to continue the RAHBO program. The 
plan is for the Department of Social Services to assume students* 
social service needs, JTGN to cover students' tuition needs, and for 
the State to pay for a project administrator. If the program 
continues, the surgical technology and certified respiratory therapy 
programs will not be included because of insufficient need for surgical 
technologists as well as the AMA requirement that certified respiratory 
therapists be observed over a three-month period, a requirement that 
was waived for persons trained under this grant. 

No dissemination activities were proposed, although substantial 
dissemination occurred through television spots, newspaper articles, 
presentations delivered at national conferences, as well as articles in 
professional journals. All three partners gave presentations at the 
following conferences: American Society for Healthcare Human Resources 
Conference, the Adult Learner Conference, and the National Conference 
on Rural Adult Education Initiatives. 
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North Clackamas School District #12 
Milwaukee, Oregon 

The grant to North Clackamas School District was used to establish 
the Student Cooperative Training Units (CTU) Program at the district's 
Owen Sabin Occupational Skills Center. The district had been 
approached by the Northwest Regional Educational Laboratory (NWREL) to 
determine if the district would be interested in applying for a 
Cooperative Demonstration Program grant. NWREL helped the district 
prepare its proposal and continued to take an active role in the 
project as both a technical support and third-party evaluator. Other 
partners in the project were Precision Castparts Corporation, Block 
Graphics Inc., and Providence Milwaukee Hospital- These partner 
organizations already had working relationships with the Center's 
instructors, and those relationships were used to create formal student 
internship positions at each company. 

The grant had seven objectives: 



■ to coordinate the development of 
curriculum for Advanced Information 
Systems, Graphics Technology, and Health 
Occupations occupational cluster programs 
to prepare students for entry into those 
fields; 

■ to provide students with practical 
training using high technology tools; 

■ to pilot the CTU Program in each of three 
occupational areas: Office Systems, 
Printing, and Health Careers; 

■ to refine and revise the curriculum as 
necessary; 

I to operate a full school year CTU 
program; 

I to evaluate the project's impact on 
participating staff, students, and 
business; and 
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■ to disseminate the project's approaches, 
processes, materials, techniques, and 
findings to state and national audiences. 

The Advanced Information Systems (AIS) cluster trained 
approximately 40 students per year in office systems using a networked 
computer terminal and a variety of word processing, electronic mail, 
scheduling, and information management software. Students enrolled in 
the second year of AIS were eligible to apply for the CTU internship at 
Precision Castparts Corporation. 

The Graphics Technology cluster trained approximately 40 students 
per year in desktop publishing, comprehensive layout, paste-up, 
masking, platemaking, offset press operation, hazardous chemical 
handling, and safety. Students enrolled in the second year of the 
cluster were eligible to apply for the CTU internship at Block 
Graphics, Inc. 

The Health Occupations cluster trained approximately 140 students 
per year in human anatomy, medical terminology, and health-related 
skills. Students enrolled in the second year of the cluster were 
eligible to apply for the CTU internship at Providence Milwaukee 
Hospital . 

Teachers selected CTU participants based on a student's maturity 
level, behavior, academic performance, and career interest. Each 
candidate was referred to a business partner, and students were 
required to follow the hiring process normally utilized by the business 
partner's company. They were interviewed by the personnel officer and 
then referred to the appropriate site supervisor for additional 
interviewing or skills testing. Students accepted as interns received 
new employee orientation, a tour of the company, and an employee 
handbook to review. Upon graduation, students were expected to enter 
the work force in that industry or continue their education at the 
community college or university. The private partner made no guarantee 
to hire students after the internship ended. Although both male and 
female students participated in the project, no minorities were 
trained. 



?77 



E-42 

The CTU program was operated in two phases: a summer pilot phase 
and a school year phase. The summer pilot phase began with an 
orientation session for interested students. Students indicated they 
would be interested in doing an internship if high school credit was 
given and if they were paid a stipend to replace the money they 
otherwise would have earned through summer jobs. Each student was 
authorized to receive up to 180 hours of training over a four- to six- 
week period. During the pilot phase it became clear that it was not 
feasible to train students in groups as had been originally proposed. 
To compensate for the lack of interaction between students on site, 
interns were required to record daily entries in journals (in order to 
improve their communication skills), and attend debriefing sessions 
where they discussed their experiences. Instructors reviewed the 
journals each week. 

During the summer pilot phase, 12 AIS students started 
internships, and all completed their training. Of the 12, six were 
hired the following year under a cooperative employment agreement (even 
though the company had a hiring freeze), five continued their 
education, and one was still looking for a job. Seven Graphics 
students started internships, and all completed their training. Of the 
seven, two were hired the following year, three continued their 
education, and one was looking for employment. Five Health Occupation 
students started internships; four completed training (the fifth 
dropped out because of a schedule conflict with a second job). None of 
the five could be hired by the partner because Oregon State regulations 
required postsecondary training to qualify as entry level technicians 
in the healthcare industry. One of the most substantial contributions 
the summer interns made was to draft, edit, and refine a new 
instruction manual for a software package (Occupational Health 
Maintenance) purchased by Precision Castparts. 

The school-year phase of the project included continued 
internships at the three partner companies. However, due to school 
classes and other student activities, students were not able to train 
for six hours per day as they had done during the summer. The training 



278 



E-43 

schedule was first modified to allow three hours per day on-site, but 
companies had difficulty with thit schedule and asked that the time be 
increased to four or more hours per day. As in the pilot phase, 
students were placed in internship positions for a period of four to 
six weeks. During the school year, six AIS students, eight Graphics 
students, and 11 Health Occupations students started internships, and 
all completed their training. 

The five teachers at the center who taught the three cluster areas 
scheduled weekly conferences with the internship supervisors at each 
company, during which the teacher reviewed each student's performance, 
learned about the impact of technology on the workplace, and became 
familiar with new equipment or industry procedures. The teachers 
integrated their new ki;owledge into their program curriculum and 
customized their courses to local industry needs. Teachers needed to 
revise their curriculum continually to fit the rapid changes in the 
industry. For example, the AIS cluster put more emphasis on personal 
computers, WordPerfect, and Lotus 1-2-3. 

Representatives from the three partners and other businesses sat 
on the advisory committee for the Skills Center (there was no separate 
grant advisory committee). The committee was asked to review revisions 
to the curriculum, and throughout the year teachers discussed with 
individual committee members any special problems they might be 
experiencing. Through the committee and through weekly conferences, 
the partners made specific suggestions about what topics they wanted to 
see included. Precision Castparts requested that excerpts from their 
employee handbook regarding grooming, attendance, and performance 
expectations be included in program orientation. Block Graphics asked 
that training in handling hazardous chemicals become a prerequisite for 
placement at its facility. Providence Hospital suggested that more 
emphasis be place on computer literacy. 
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Partners for American Vocational Education (PAVE) 
Alexandria, Virginia 



Partners for American Vocational Education (PAVE) is a private, 
nonprofit foundation organized to create partnerships between business, 
industry, and education. PAVE managed the Cooperative Demonstration 
grant, which consisted of partnerships established between a coalition 
of employers and a coalition of educational institutions. PAVE is in 
Alexandria, Virginia, and most of the employers and educational 
institutions involved in this gr?nt are located in nearby Washington, 
D.C. 

This project, called Business-Education Venture for Health Care 
Occupations Training, had two primary goals: 

■ to develop and implement an effective Busi- 
ness-Education Venture that will maximize the 
resources of educational institutions cn 1 
health care providers in the District of 
Columbia to train and place skilled tech- 
nicians in the health care industry; and 

■ to field test a health care high technology 
Business-Education Venture that will enroll 
150 persons in training programs that will 
provide them with skills to increase their 
value, performance, and employability as 
technicians in the health tare industry. 



Student training was conducted in five areas for health care workers: 



I nurse occupations technologies, which 
provides a minimum of 75 hours of lab work 
and on-the-job training: 

■ medical transcription technology; 

■ phlebotomy technology; 

I medical records apprenticeship; and 

■ medical unit clerk courses. 
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These training programs typically were six to nine hours per week 
for 12 to 14 weeks* Cost per student ranged from $25 to $150 ♦ 
Participants registered by telephone through the PAVE-run Health Care 
Training Hotline* Prior to training, students were assessed using 
Valpar MESA Short Form, an assessment software package* 

Approximately 71 students, half of whom were already employed, 
completed specialized health care training provided by PAVE* 
Approximately 75 percent of those students not having jobs upon 
entering the programs found employment upon completing the program: 

I one hundred percent of 18 phlebotomy 
technology completers; 

■ seventy-one percent of 35 nurse occupations 
technologies completers; and 

■ seventy- two percent of 18 medical 
transcription technologies completers. 

The medical records apprenticeship and medical unit clerk courses 
did not yield certificates of completion; 27 students took part in 
these courses. Fifteen students received some degree of remediation, 
which PAVE managed using BASE software. 

One unforeseen problem PAVE experienced was the extensive "fall 
out" between the time initial contact was made and participants 
actually were enrolled in training: although 255 students initially 
contacted PAVE, only 196 enrolled in training. And, a high number of 
students (35 percent) dropped out of programs before their training was 
completed. 

PAVE's relationship with its partners was different from other 
Cooperative Demonstration grantees for several reasons. First, PAVELS 
role was to manage partnerships. It oversaw the partnerships formed 
between the coalitions of health care employers and educational 
institutions mentioned earlier. The coalition of employers included 
the Washington Hospital Center, Howard University Hospital, Greater 
Southeast Hospital, Childrens* Hospital, Little Sisters of the Poor (a 

2n 



E-46 

long-term care nursing home), and the Washington Nursing Facility. The 
educational institutions that provided training included the District 
of Columbia Public Schools and the University of the District of 
Columbia. Thus, PAVE established many different networking 
partnerships with each of these institutions, and it facilitated 
training between them. This third-party management feature made this 
program unique. 

This grant also was unique in that PAVE took over the role of 
referring students to existing training programs, although some of 
these programs were adapted to meet the needs of PAVE trainees. PAVE's 
role was to recruit students for existing training, instead of the more 
common Cooperative Demonstration grantee role of actually providing the 
training. 

High-technology aspects of this program were the nursing 
procedures and technologies used, particularly in the phlebotomy 
technology and nurse occupations technologies training programs. 

Dissemination of program activities were to include a guide on the 
development of a Business-Education Venture, a manual to facilitate 
model replication as outlined in the proposal. Instead, a briefer 
version of this manual, with a broader focus, was developed in the form 
of guidelines. These were disseminated to State directors of 
vocational education and State councils on vocational education. 
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Southwestern Community College 
Chula Vista, CA 

Southwestern Community College (SWCC) received a Cooperative 
Demonstration grant to design and implement a Comprehensive Aerospace 
Manufacturing Technology Program. As originally proposed, the college 
was to develop a program that would train disadvantaged individuals for 
jobs in aerospace; upgrade skills of current aerospace employees to 
maice them more promotable; and bring together high schools, four-year 
colleges (San Diego State University), and companies (especially Rohr 
Industries) through Southwestern Community College. The project 
planned to serve three groups: local high school students, existing 
SWCC students and persons not employed in the aerospace industry, and 
industry upgrade trainees (employees). There were four major program 
components: recruitment, assessment and placement in appropriate 
educational /training program, training (basic educational skills, core 
aerospace skills, and advanced technical skills), and student/trainee 
support services and job placement. 

As delivered, this project appeared to provide some services to a 
number of different groups, but there was little relationship among the 
services. Each of the three target groups received some service. It 
is hard to determine which services the project initiated from scratch 
and which were added to services, such as mentoring, tutoring, and 
counseling, already offered at SWCC. The thread that, appeared to run 
through the overall effort was an attempt to reform instruction (or 
build capacity) at SWCC. 

The downturn in defense procurements wiped out the growing need 
for aerospace employees at all levels. Rohr Industries, slated to be 
the primary partner, laid off one-third of its employees. The Rohr 
employee who was to be the full-time industry liaison for the project 
lost his job. In addition, the project encountered problems in 
staffing. The proposed project coordinator could not wait for an 
uncertain grant. It took five months to find a new coordinator. 
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Despite these problems, the project did accomplish several 
objectives. It carried out tutoring, curriculum development, revision 
of the engineering technician and engineering transfer programs, 
developed workshops, placed interns, and linked the institution to 
NovaNet, a system that allows students access to thousands of 
interactive, educational software packages, many of which teach 
remedial n^ath and science. The project coordinator was able to 
establish limited partnerships with other companies. 



i Tutoring . The project supported after-school 
tutors in high school math and science during 
the 1990-91 school year. The project esti- 
mates that 250 students received some 
tutoring. 

■ Mentoring . Forty-one SWCC and 17 high school 
students were contacted by and/or partici- 
pated in a mentoring program sponsored by the 
campus* chapter of the Society of Hispanic 
Professional Engineers (SHPE). 

I Competency-Based Instruction . Ninety-eight 
SWCC students participated in classes in 
which instructors had received stipends for 
curriculum development aimed at making the 
courses competency based. 

■ Career Counseling . Twenty-six SWCC students 
attended job placement workshops, 16 produced 
resumes, 30 attended a Careers in Engineering 
workshop, 87 attended a career counseling 
overview session, and 44 obtained one-on-one 
counseling session at the Career Center on 
campus. 

■ Remediation . The project installed five 
terminals on the NovaNet system. Approxi- 
mately 100 students used the remedial 
software. A number of SWCC students improved 
their scores on an algebra readiness test 
after working on the NovaNet remediation 
system. 

■ Internships . Despite the downturn in the 
local aerospace industry and the general 
economy, the project coordinator was able to 
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place several students in internships at 
local companies ♦ Six summer positions were 
developed in 1991. There are currently 11 
industry requests for 1992 ♦ 

■ Classes at Rohr . Five SWCC classes were held 
at Rohr for Rohr employees ♦ A total of 87 
employees participated. In addition, 55 Rohr 
employees received some counseling, and 14 
used NovaNet* 



As for the high-technology focus, the project planned to train 
students and workers with varying skill levels to work in the aerospace 
industry. Coursts already available at SWCC dealt with manufacturing 
engineering, computer-aided manufacturing, industrial engineering, and 
quality engineering. 

Given the extremely local nature of the services and the 
difficulties this project encountered, it is not clear what elements 
are exportable to other sites. Some of the curricula developed at SWCC 
may be usable by others, and other institutions may want to look into 
sweeps experience with the NovaNet remediation system* 
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State Center Community College District 
Fresno, California 

State Center Community College received a Cooperative 
Demonstration grant of $399,000 to establish an Advanced Technology 
Center and provide high technology manufacturing training to 
disadvantaged populations. The proposal envisioned the development of 
an Advanced Technology Center (ATC), complete with new facilities and 
curricula, located in downtown Fresno, The ATC would recruit local 
unemployed, disadvunUjed, and refugee populations through social 
service agencies and train them in high-tech skills needed by area 
manufacturers. Courses were to cover topics such as Industrial 
Controls, Hazardous Materials, Management of Manufacturing Processes, 
Just In Time (JIT) Inventory Controls, Computer Aided Design (CAD) and 
Computer Aided Manufacturing (CAM) and NC Programming, Courses were to 
be taught in short but intense modules (9 weeks, 20 hours per week). 
After completing the courses, trainees were to be placed in three-month 
internships at companies represented on the project advisory board. 
The project would create a Manufacturing Technician Certificate Program 
and establish an articulation agreement with California State 
University at Fresno, After the project, the ATC would serve primarily 
as a short-term technical training center for the employees of local 
manufacturers. 

As delivered, the project established an ATC and developed 
curricula in Hazardous Materials, Computer Assisted Design (CAD), 
Computer Automated Manufacturing (CAM), and industrial electronics 
(including the operation of programmable logic controllers). The ATC 
is located on the campus of Fresno Community College, rather than 
downtown, because the promised warehouse space never materialized. 
Furthermore, there has been little training of unemployed or 
disadvantaged populations. According to project staff, representatives 
of companies on the Advisory Board insisted on levels of training 
beyond what could be handled by most disadvantaged students. Most of 
the training went to employees of local manufacturers. The ATC (both 
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the facility and the curricula) were incorporated into the college's 
Industrial -Tech. Department at project end. 

This project had several accomplishments: it acquired a wide range 
of high tech equipment for training at the ATC, it gave instructors 
support to develop and deliver the ATC curricula, and it strengthened 
the college's relationships with local manufacturers by allowing them 
to play a role in curriculum development and by training their 
employees (with grant funds). 

Much of the training provided by the project was equipment- or 
software-specific. For example, students/employees took a class to 
learn how to operate a particular piece of equipment, such as a 
programmable logic controller (PLC). The original design of the 
classes as nine-week modules required this high level of specificity. 
In several classes, however, this nine-week design was changed. In the 
Hazardous Materials classes, the nine-week time period was simply too 
short to learn about all the different materials and regulations. In 
reality, the classes -became more like normal college classes-3 hours c 
week for 18 weeks (one semester). According to project staff, the 
scheduling of the classes was often determined by the company 
representatives on the advisory boards, and was designed to match up 
with the workshifts at local companies. For instance, PLC classes were 
taught in the early morning so that employees could attend before work. 

In several of the classes, such as PLC and CAD, instructors 
discovered midway through a course that a number of students/employees 
lacked the basic math and electronic skills necessary to complete the 
class. The instructors had to stop teaching the specific skill 
(equipment or software) and teach the basics. (Several industrial 
electricians did not know what ohms and amperes measured, and several 
drafting students were not proficient in trigonometry.) 

As part of the incorporation of the ATC into the college's 
Industrial-Tech department, the college is working to develop a broader 
two-year certificate program in Advanced Technology. All students in 
this program will take core classes in math and electronics, then 
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select a specialty in Hazardous Materials, industrial electronics, or 
CAD/CAM. 

A total of 124 persons received training. Of these, 98 were 
current employees of companies. A variety of courses were offered, 
including CNC control (first class 20 hours, second class 45 hours-24 
students registered and 22 completed) » Basic Programmable Mill (first 
class-20 hours, second class 45 hours-*18 students registered and 
completed training), CNC Manual Programming (1 class, b students, 90 
hours of training). Tool Design (1 class, 8 students, 90 hours of 
training), Basic Programmable Lathe (7 students, 45 hours), CNC 
Programming (9 students, 45 hours of training), Intro, to Hazardous 
Materials (2 courses, 54 hours each, 14 students each), and Industrial 
Hazardous Waste Treatment (2 courses, 54 hours each, a total of 16 
students). 

As for exportability of what was learned in this project to other 
sites, the original proposal noted that as of February 1989, 41 ATCs 
had been established in 18 states. The project director pointed out 
that most, these ATCs are in hard-hit industrial areas. Fresno is 
largely an agricultural area, where the potential users of high- 
technology equipment are in food-related industries such as processing 
and packaging plants. The establishment of the ATC is also part of an 
effort to attract non-agricultural industry to Fresno. 
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Uaubonsee Comnunity College 
Sugar Grove, Illinois 

The office technology program, "Survival Skills for Office 
Technicians'*', has been operating at Waubonsee Community College (WCC) 
since the late 1970s and has been supported by a variety of public 
funding sources. The program has been distinct from the regular, for- 
credit office careers training at WCC both in sources of funding and 
philosophy of operation. Where postsecondary students would pay to 
enroll in office careers classes and receive instruction in a 
traditional classroom structure, students in the office technology 
program had their training costs paid by another agency and worked at 
their own pace. The program also was distinct from the customized 
training offered to local employers. Prior to WCC's first Cooperative 
Demonstration grant in FY 1988, most of the noncredit office technology 
training was supported by CETA and. Job Training Partnership Act (JTPA) 
funds through the Kane-DeKalb County Private Industry Council (KDC- 
PIC). 

WCC used its first Cooperative Demonstration grant to provide 
basic skills assessment, training, and job placement to 208 minority or 
disadvantaged women. The first grant provided training in basic 
keyboarding, filing, bookkeeping, switchboard operation, office 
decorum, database management, word processing, and electronic 
spreadsheets. The seco^id grant (FY 1989) added training in Unix-based 
software (SAMNA and Lotus), electronic mail, electronic shorthand, fax, 
and desktop publishing. The second grant also enabled the instructors 
to place more emphasis or computers. For example, the typing course 
went from using manual typewriters to computer.-, and the filing course 
went from using a manual file box to a computerized filing system. 

Under both grants, classes were held at WCC*s downtown Aurora 
campus from 8:30 a.m. to 4:00 p.m. Monday through Friday (with two 15- 
minute breaks) to simulate the actual office schedules. Evening hours 
were added during the first grant period to accommodate those people 
already working, but students had to commit to at least 12 hours of 
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instruction per week. The program is continuing in its entirety after 
the end of the grant* 

Although the "Survival Skills for Office Technicians" program is 
an outgrowth of an existing program for JTPA clients, students are 
eligible for training only if they are net eligible for JTPA. The 
exclusion of JTPA clients is in part to satisfy concerns of the KDC-PIC 
about competing for clients, but also in part specifically to serve 
those women not being served by JTPA. For example, WCC was aware of 
widows and divorcees who would have had to sell their houses in order 
to be eligible for JTPA. The grant was designed to assist these women 
as well as others with no family income or assets. Child care was 
provided on campus and paid for by the project if necessary. 
Approximately 94 percent of the people trained under the second grant 
were women and 49 percent were minorities; 24 percent were unemployed 
at the time of training. 

Students were referred to the program through private employment 
agencies, partner organizations, and the WCC counseling office. 
Project staff advertised the program and classes through church groups, 
the Urban League, the local Migrant Council, and other community-based 
organizations; staff were approached by a prison-release program to 
include those clients, as well. Students admitted to the program had 
to have a minimum of a 10th grade reading/comprehension level and a GEO 
or high school diploma or be in a GEO class. Basic skills levels were 
assessed through ASSET tests given by the college's assessment center. 
There also was a two-day assessment in the Office Technology Department 
for pretests in math, English, spelling, filing, and typing. 

The partners in the grant were companies that had either referred 
individuals to the program in the past or who had hired people trained 
by the program. For example, one of the 12 partners was the Illinois 
Department of Employment Security (IDES), which referred people filing 
for unemployment to WCC for retrainT^^;. The project director would 
certify to the IDES those students being trained each week so the 
students could continue to collect benefits. Another partner was the 
Kane County Circuit Court, which had hired students from the program in 
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the past. A third partner was the Kane County Department of Public 
Aid, which both referred people to the program under its own "Project 
Chance" and which hired graduates of the program^ There were no formal 
partnership agreements between WCC and the private groups, and the 
project had no formal advisory board of partners. 

The program included an unpaid work-experience component for those 
students who could type 30 words per minute and who had some word 
processing skills. Approximately 15 percent to 20 percent of the 
students were placed in internships at any one time. The project 
director arranged for placements in the partner organizations. 

All of the seven project staff members were part-time (35 
hours/week) employees of WCC and all were paid entirely from grant 
funds. As part-time employees, staff received no fringe benefits other 
than retirement. The lack of benefits made it difficult for the 
project to attract and hire staff. This lack of benefits made staff 
members even more aware of the iiriportance of placing students in jobs 
with benefits. Consequently, the staff made an extra effort to find 
students jobs with public agencies. 

Of the 40 courses offered by the program, 15 were new courses 
during the second Cooperative Demonstration grant. The new courses 
were on subjects either suggested by partner organizations (e.g., 
customer service) or by student feedback on their work experience 
(e.g., proofreading). The new courses were developed by project staff 
using commercially available products (e.g., a proofreading curriculum 
package) or from portions of software manuals and supplementary texts. 
The office telecommunications course was developed by an intern from 
Indiana University, who was paid only for her teaching time but who 
received college credit for developing the course. The staff purposely 
did not look at courses from the other colleges because the staff felt 
that the students being trained needed an approach different from that 
used in for-credit courses. 

The open-entrance\open-exit and self-paced structure for the 
training are available for export but are not well documented. Copies 
of curriculum packages are available to other institutions upon 
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request. Northern Illinois University has implemented the packages for 
WordPerfect, keyboarding and data entry, and customer service. Two 
local school districts are reviewing the packages for possible use. 

The project director has arranged for three articles about the 
project in the local newspaper and appeared on two TV shows. The 
newsletter for all Cooperative Demonstration grantees that was started 
under the first grant was discontinued after four issues in the second 
grant. 



392 



E-57 



West Virginia Department of Education 
Charleston, West Virginia 

The West Virginia Department of Education, Bureau of Vocational, 
Technical, and Adult Education Is located In Charleston, West Virginia* 
This grant was unique In that the grantee oversaw two training programs 
offered by separate vocational centers within the state: the Marlon 
County Vocational-Technical Center and Carver Career and Technical 
Education Center* These programs, which were Implemented as a result 
of the Cooperative Demonstration grant. Involved training In two 
different technology areas: health and computers* The grant programs 
were located In Charleston and Fairmont, West Virginia, about two hours 
apart by car* Roy Thomas was the overall project director and 
coordinator, and there were local project directors at each site* 

Specific objectives of the project, as stated In the proposal, 
were to: 

I Implement cooperative training programs In 
the occupational areas of ADA computer 
language use, respiratory therapy, and 
hospital pharmacy technician; 

■ train or retrain the unemployed, the 
underemployed or other adults who need 
upgraded skills In these technical areas; 

I place 80 percent of the trainees In jobs for 
which they have been trained at the end of 
the project; 

I Identify, develop, or adapt Instructional 
materials for program use; 

I produce process and evaluative reports at the 
end of the project; and 

I disseminate project activities and results on 
a State and national level* 

Marlon County Technical Center (MCTC), located In Fairmont, West 
Virginia, developed and administered the ADA computer language program* 



E-58 

This program became a part of MCTC's existing robotics/automated 
technology program. An extensive curriculum, "A Secondary/ 
Postsecondary Curriculum for the ADA Programming Language" was 
developed as part of the grant. The curriculum as originally proposed 
would serve only adult technical students and industry personnel; 
however, as developed, it serves secondary students as well. 

Another change in the original pUn for this program was the 
insufficient knowledge-base of members of local industry. They had to 
be taught the value of ADA if the training was to have any value to 
them. Their lack of familiarity with DOS was a hindrance to using ADA. 
Program staff, therefore, taught "DOS for Managers" and developed a 
manual to accompany the training workshop. 

Marion County teamed with J&S Machine Corporation, which donated 
equipment and acted as a liaison to local industry. Ties with West 
Virginia University and Fairmont University, which assisted in ADA 
language curriculum development, also were established. Four business 
and industry workshops were taught during the project, two ADA 
workshops were held for teachers across the State, and one ADA 
teleconference for teachers also was held at Fairmont State University. 

ADA courses offered at MCTC were held for 18 six-hour sessions 
during a six-month period. The ADA program, which employed one new 
teacher who also was the main curriculum developer, proposed to serve 
10 students at MCTC and 10 business and industry personnel. This goal 
was met, although training provided to 34 business and industry workers 
was not in ADA as originally proposed, but in other computer 
technologies and computer-assisted drafting and manufacturing. ADA in- 
service training, however, was provided to 30 teachers. Eight students 
completed the MCTC training. 

Carver Career Center developed pharmacy technician and respiratory 
therapy technician training programs. The idea for these programs was 
already in motion, although actual partnership had not begun prior to 
the grant. Carver teamed with six health care providers, including the 
Charleston Area Medical Center (CAMC), which donated equipment and 
staff to serve on the advisory board. Existing curricula-from the 
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California College for Health Sciences and the Michigan Pharmacists 
AssociationHrfere adopted with minor adaptations* A library and 
laboratory were set up at Carver for the respiratory therapy program, 
and Carver has an application pending for national certification of 
this program as well as State certification for the pharmacy technician 
program. 

An eleven-month instructional program was conducted for the 
respiratory therapy program, which consisted of both classroom and 
clinical experiences* Students in the pharmacy program chose one of 
two five-and-a-half month training programs, which also involved 
classroom as well as clinical components* 

The goal for the respiratory program, which employed two new 
teachers, was to train 30 students* Thirty- four students enrolled, and 
24 of them completed the program* The goal for the pharmacy program, 
which employed one new teacher, also was to train 30 students. Twenty- 
nine students were enrolled, and 23 completed the program* 

Dissemination activities of the grant included one television 
report and five newspaper articles. ADA workshops and curriculum 
guides, as mentioned above, contributed significantly to successful 
dissemination practices. The Carver Career and Technical Center staff 
developed and distributed recruitment and promotion brochures of the 
health care programs* Roy Thomas made presentations at conferences of 
the National Association of State Directors of Vocational Education, 
the American Vocational Association Conference, and at the annual 
conference of the National Association for Program Improvement in 
Vocational Education* Copies of the final report and all curriculum 
materials are being presented to all vocational directors in West 
Virginia as well as to the National Center for Research in Vocational 
Education, the ERIC system, the East Central Curriculum Center, and the 
West Virginia Curriculum Center. Copies of the final report are being 
distributed to all State directors of vocational education. 
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West Virginia Northern Community College 
Wheeling, West Virginia 

West Virginia Northern Community College (WVNCC) is a two-year 
college located in Wheeling, West Virginia, with three branch campuses, 
including one in Weirton* Weirton, in the heart of steel and coal 
country, is the home of Weirton Steel Corporation, the seventh largest 
steel company and the only employee-owned steel firm in the United 
States* Twenty percent of all Weirton residents work in the steel 
mill. 

The partnership goals were to develop a curriculum and provide 
upgraded training for current plant workers. The training included 
high-technology computer education for workers that would enable them 
to operate a new data collection system at the plant. Training also 
would include craft instruction in high-tech skills for pipefitters and 
millwrights so they could operate new, automated equipment for hot 
strip mill renovation at Weirton. 

The new data collection system that became the focus of the 
training was the Integrated Mill Information System (IMIS), purchased 
froii; Computer Services Corporation. IMIS was purchased to better track 
products being manufactured in the mill and more quickly ascertain 
their production status in order to enhance customer service. Due to 
problems in implementing IMIS, plans changed from training plant 
workers to conduct statistical and other advanced analyses to training 
them in data entry and initial computer orientation. Although the 
training program was essential for workers to learn IMIS, similar 
training with newly implemented information systems typically is 
conducted by the vendor. 

Classroom training took place at three-hour sessions for three 
days at Weirton Steel's classroom facilities, after regular work hours. 
One crew (per mill operating unit) was trained at a time. Employees 
received on-the-job-training for about two weeks, although more was 
available if needed. Weirton Steel compensated all 1,200 employees who 
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received IMIS training in the form of five hours of vacation time for 
every three hours of classroom training, as well as free meal tickets. 

Curriculum development was a major undertaking but an essential 
part of the work of this program. Forty- four curriculum modules were 
developed for IHIS training, one module for each machine at Weirton 
Steel. Staff development also was extensive. Trainers were carefully 
screened and, when hired, trained for 40 hours a week for six weeks. 

The craft training component of the grant was a much smaller 
dimension than originally specified in the proposal. Workers in the 
hot mill were vendor-trained, with WVNCC trainers playing only a 
minimal role. Craft training was given to 389 hot mill workers. 
Topics of craft instruction were hydraulics, lubrication, shaft 
alignment, welding, pipefitting, and scaffolding. Classes were offered 
for eight-hour sessions during the work week. To complete the 200-hour 
craft training program, employees attended class 40 hours per week for 
five consecutive weeks. 

The relationship between WVNCC and Weirton Steel was not a new 
one. They had already formed a partnership in 1987 for the Workplace 
2000 Workplace Literacy Program. This training program was a new one, 
however, and in order to run it seven full-time an-i five part-time 
trainers were hired. Also user support team and plant floor support 
team members-all Weirton Steel employees-facilitated on-the-job use of 
classroom learning. 

Dissemination activities included a national teleconference called 
"Education: Bridging the Gap," which included topics such as the 
business/industry partnership, employees' fear of change, and assessing 
training effectiveness. WVNCC staff made presentations in Florida-at 
the Leadership in Education conference, Florida State University's 
national conference, and in California-at the International Conference 
for the Council for Adult and Experiential Learning. 
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SUMMARY OF SURVEY OF FY 1989 PROJECTS 



This appendix presents detailed findings from a survey of the FY 
1989 grantees under the Cooperative Demonstration Program (High 
Technology) of the 1984 Carl D. Perkins Act, All 30 FY 1989 grantees 
were sent the survey, and, as of November 1991, responses (ir^jluding 
telephone data retrieval) were obtained from 26 grantees. The grantees 
responding to the survey are identified with an asterisk on the list of 
FY 1939 grantees presented in Appendix B-2. In general, respondents 
were the individuals responsible for day-to-day management of the 
project. Although 26 grantees returned questionnaires, results 
reported here usually total 27 because one grantee was a state 
education agency that, in turn, dispensed the funds to two, unrelated, 
projects. 

The survey was designed to collect information on project 
implementation, especially factors related to start-up, goals, major 
activities, partner involvement, and levels and types of services. 
Survey questions pertaining to project costs were discussed in the 
cost-benefit analysis in Section IV. Many of the survey questions were 
generated from findings of case studies in eight FY 1988 grantees 
visited during the 1990-91 school year. In addition to describing 
programs, the questionnaire was designed to provide a framework for 
subsequent site visits. 

Instructions for the survey were distributed in stages to 
respondents. In early February 1991, all project directors received a 
detailed memorandum from the Office of Vocational and Adult Education 
(OVAE) indicating the information about clients, . staffing and extent 
training that would be requested in the questionnaire. The memo was 
followed by phone calls from the study team to all grantees over the 
next month to reiterate the need to obtain the data and to answer any 
questions grantees might have about the memo. The survey was mailed to 
grantees in July 1991, and telephone calls were made to obtain 
responses continued throughout the summer and fall« 
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Despite the early memo and phone calls, and repeated telephone 
follow-ups to the survey mailing, only 26 of the 30 grantees returned 
questionnaires, and about one-third of the respondents provided 
incomplete systematic information.* As a result, it is not possible 
to provide findings for the course completion and student outcome data 
requested by the survey. To facilitate the narrative discussion, 
simple frequencies are not presented as tables in the text. 

Training Areas 

The legislation mandating the Cooperative Demonstration Program 
allowed for projects in a wide range of industries, but most of the 
projects that were funded under the demonstration offered training in a 
few industries. Manufacturing was the most common industry focus, 
although health care and business services also were training areas 
(see questionnaire, item 7). Seventeen of the respondents reported 
manufacturing to be an area in which students were trained, seven 
projects indicated health care to be an area of training, and five 
reported a business services focus. Seven respondents indicated that 
training took place in multiple industries, and to specify a few 
industries would be misleading.^ 

Projects were divided almost equally between those that saw 
training of students to be their "key" activity and those that chose 
various activities associated with the development of the training to 
be most important (see questionnaire, item 6). Thirteen grantees 
identified student training itself to be the most important project 
activity, while five selected job skills identification, and five 
selected curriculum development as the dominant activity. Student 
recruitment, staff development, industry/occupational change, and 
institutional capacity building each were selected by one respondent. 

All projects provided occupation-specific training, and the 
majority (16 of 27) provided non-occupation-specific training as well 
(see questionnaire, items 26 and 31). The most common forms of non- 
occupation-specific training were provision of employability skills (13 
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projects), provision of basic academic skills, (11 projects), and 
provision of advanced academic skills (nine projects). 

Offerings were designed to teach both entry level and more 
advanced skills (see questionnaire, item 8), Although eight grantees 
reported that training was geared entirely to entry-level work and five 
reported that it was geared entirely to upgrading skills, 14 grantees 
reported that training was designed for both. Grantees providing 
training in health care were slightly more likely to emphasize entry- 
level skills and slightly less likely to emphasize upgrading or both 
types than other grantees. None of the programs training in business 
subjects taught entry-level skills entirely (see Table F-1), 

Overall, projects were somewhat more likely to be stand-alone 
efforts than parts of existing training programs at grantee 
institutions (see questionnaire, item 9), Eight grantees indicated 
that their projects were part of ongoing training efforts, while 14 
indicated that the grants supported separate training programs. An 
additional five grantees described their efforts as both stand alone 
and parts of ongoing training programs.^ Projects training in 
construction, manufacturing, or multiple industries were somewhat more 
likely than those training in other fields to be separate from regular 
institutional offerings (see Table F-2), Ten of the 17 programs that 
indicated manufacturing as an area of training also indicated that the 
training was separate from other offerings at the institution. 
Although they may not have been tied to ongoing offerings, 
projects were likely to serve groups that previously had been served by 
the institution (see questionnaire, item 10), Only five grantees 
reported that the project served a new or different group exclusively, 
and eight indicated that the project was intended to benefit groups 
similar to those served previously. Fourteen projects indicated that 
both new and similar groups benefitted. Projects training in business 
services were the most likely to indicate that new groups benefitted 
exclusively (see Table F-3), Projects that provided training in health 
care were the most likely to indicate that groups similar to those 
served previously were served through this project. 
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Table F-1 

MAJOR INDUSTRIES OF TRAINING AND SKILL LEVEL OF TRAINING PROVIDED 

(NUMBER OF PROJECTS) 



Number of 
Industry* Projects* 


Entry-1 evel Upgrade 

Skills Skills Both 


Construction 4 
Manufacturing 17 
Business Services 5 
Health Care 7 
Education 4 
Multiple Industries 7 


0 1 3 
3 3 11 

0 1 4 
3 1 3 

1 1 2 
0 2 5 



Industries with fewer than four responses omitted from analysis. 
Adds to more than 27 because projects could select more than one 
industry. 
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Table F-2 



MAIN INDUSTRIES AND PLACEMENT OF PROJECT 
WITHIN GRANTEE INSTITUTION 
(NUMBER OF PROJECTS) 



Number of 
Industry* Projects* 


Addition to Separate 
Ongoing Training 
Trai ni ng Program Both 


Construction 4 
Manufacturing 17 
Transportation 4 
Business Services 5 
Health Care 7 
Education 4 
1 Multiple Industries 7 


0 4 0 
3 10 4 

1 2 1 

2 1 2 
2 4 1 
2 2 0 
1 5 1 



* Industries with fewer than four responses omitted. 

Adds to more than 27 because projects could select more than one 
industry. 
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Table F-3 

MAIN INDUSTRIES AND SERVICE TO 
NEW OR SIMILAR INSTITUTIONAL CLIENTELES 
(NUMBER OF PROJECTS) 



Number of 
Industry* Projects* 


Proiect Benefits 

Similar 
New or Group to 
Di f f erent Previ ously 
Group Served Both 


Construction 4 
Manufacturing 17 
Transportation 4 
Business Services 5 
Health Care 7 
Education 4 
Multiple Industries 7 


1 2 1 
4 2 11 

1 0 3 

2 1 2 
2 4 1 

1 I 2 

2 1 4 



* Industries with fewer than four responses omitted. 

Adds to more than 27 because projects could select more than one 
industry. 
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Projects characterized the likely "rewards" of training for 
participants as occupational in nature (see questionnaire, item 65). 
According to the grantees, the most common student reward was likely to 
be a job in a particular field (22 projects). Beyond that choice, 
however, the most commonly given response was not on the questionnaire. 
Nineteen respondents wrote in responses that identified some form of 
behavioral training objective— most saying that students would be able 
to perform a particular set of occupational skills. Promotions in a 
particular field or with a particular employer were anticipated by 1? 
and 15 projects respectively. Formal education credentials were 
somewhat less popular than possible student rewards, with 14 projects 
anticipating vocational certificates, and 11 each selecting a 
degree/diploma or acceptance into further education. 

From an open-ended question on project beneficiaries, the study 
team organized projects by several primary types or classes of 
participants. These types were not mutually exclusive but were 
intended as general characterizations of project clienteles. They 
included: current employees of companies (13 projects), adults seeking 
work in a field or company (seven projects), disadvantaged adults (four 
projects) and high school students (three projects). For six projects, 
a second-level client was also characterized. Two of these projects 
indicated that second-level beneficiaries included regular college 
students, two indicated high school students, and one each indicated 
schoolteachers or disadvantaged persons. Two projects also identified 
a third type of client. 

Projects providing training in manufacturing were the most likely 
to indicate current employees as their primary clientele (see Table F- 
4). Fifty-nine percent (or 10 of 17) of such projects indicated that 
current employees were their primary clients. Programs providing 
training in multiple industries also indicated that they served 
existing employees primarily (four of seven, or 57 percent). Projects 
providing health care training were more likely to indicate that they 
served adults seeking work in a field or company (three of seven), 
which fits with the earlier finding that such projects were somewhat 
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more likely than others to provide entry-level training exclusively. 
Only one project providing health care indicated that it served current 
employees of companies. Projects offering training in business 
services were somewhat more likely than others to indicate that they 
served disadvantaged adults (two of five projects)/ 

Projects training for manufacturing and multiple industries were 
also more likely to train persons employed full time (see Table F-5). 
Although information on employment status of clientele was not 
available for all such projects, of the 16 projects training students 
in manufacturing for which data exist, seven indicated that 76 percent 
to 100 percent of students were employed full time, and an additional 
three indicated full-time employment by 51 percent to 75 percent of 
students. In contrast, none of the six projects providing training in 
health care for which information is available indicated that 76 
percent to 100 percent of students were working full time, and only one 
indicated that 51 percent to 75 percent were working full time. 

From these findings, then, a rough pattern of projects begins to 
emerge. The most common offerings appear to be in the area of 
manufacturing. Training provided under the grant is relatively 
separate from regular offerings at the same institutions; the clientele 
typically is current employees (most likely full- or part-time 
employees in the fields to which the project is geared); and this 
population (current employees) is one that the institution has served 
previously. Training appears to be geared to achieving mastery of a 
specific set of occupational skills more than to obtaining formal 
credentials. 

Economic Conditions and Effects on Pro.iects 

From reviewing the project files and the site visits to FY 1988 
grantees, the study team anticipated that training aimed at jobs or 
promotions would be a major part of grantee activities. Project 
proposals often included job placement goals for participants, so the 
ability to find jobs for trainees would be of paramount importance. As 
a result, the questionnaire asked grantees about the economic climate 
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in the communities in which they were located as well as in the 
specific fields for which training was provided. 

Grantees indicated that economic conditions in their communities 
had deteriorated since they applied for grants (mid-1990), and most 
indicated that the changes had affected implementation of the grant 
(see questionnaire, items 1 and 2). About half the grantees (14) 
indicated that economic conditions in the community had deteriorated, 
and 11 grantees indicated that conditions re'.nained about the same 
throughout the period. [When asked whether the decline had affected 
grant implementation (e.g., did it mean fewer jobs for students or 
problems for the institution?), 10 of the 14 who acknowledged 
deteriorating economic conditions indicated that this had an effect on 
grant implementation. Four said that the changes had resulted in fewer 
jobs for students, and four indicated that the changes resulted in 
problems for the institution.] 

Grantees were slightly less likely to indicate that a decline in 
economic conditions in the specific industry or occupation in which 
training was provided had affected project operations (see 
questionnaire, items 3 and 4). Well over half the respondents (16) 
indicated deteriorating economic conditions in the specific fields of 
training, but only half of those indicating a negative economic change 
also reported that it affected project operations. Six of these cited 
poorer job prospects for students as an outcome, while three indicated 
a negative effect on the institution. Two grantees reported that 
negative changes in the industry created a positive opportunity for the 
project by increasing training needs, and one grantee indicated that 
better economic conditions in the industry led to fewer training 
opportunities at the institution. These last responses may reflect a 
common finding that bad economic times, although creating poor job 
prospects, can sometimes translate into increased enrollment in 
training programs. Not only do the unemployed seek training, but 
persons who are concerned about losing their jobs may also enroll. 

Deteriorating economic conditions appear to have been most common 
among the projects that provided training in manufacturing (see Table 
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Among the grantees, 71 percent of those that provided training 
in manufacturing reported that economic conditions in the industry of 
training had deteriorated since the grant application. Among those 
grantees providing training in health care, only 29 percent reported 
deterioration, while among those providing business services, 40 
percent indicated economic deterioration in the industry* Because the 
individuals enrolled in projects offering manufacturing training were 
the most likely to be full-time workers (see Table F-5), however, the 
economic deterioration may not have translated into immediate job 
effects. 

Pro.iect Goals and High-Tech Focus 

In the first year of the evaluation, site visits revealed that 
grantees had a wide variety of goals, with a substantial subset of 
grantees concerned as much with curriculum, staff, or institutional 
development as with the direct provision of training. For FY 1989 
grantees, there was a major change in Federal direction: only 
applicants promising a "high-technology" training focus were considered 
for awards. It was thought that this constraint might narrow the range 
of project goals, so several questions were included in the survey in 
an attempt to understand the relative importance of training and other 
goals as well as the nature of the "high-tech" focus itself. 

Once again, the grantees expressed a wide variety of goals. For 
the FY 1989 grantees, provision of training was the single most 
important goal, but almost half the respondents selected another goal 
as most important (see questionnaire, item 5 and Table F-7). Of the 27 
respondents, 15 indicated that creating new or improved training was 
the first goal. An additional seven grantees reported that increasing 
access to training for special or otherwise underserved populations was 
most important (see Table F-8). Given the overall Federal objective to 
develop models of public/private cooperation, it was surprising that 
the establishment of partnerships was ranked first by only three 
respondents. Improving economic productivity was the primary goal of 
two grantees.^ 
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Table F-6 

MAIN INDUSTRIES AND LIKELIHOOD OF ECONOMIC CHANGE OVER PAST TWO YEARS 

(NUMBER OF PROJECTS REPORTING) 



Number of 
Industry* Projects* 


Industry Conditions 

Remained 

Improved Deteriorated the Same 


Construction 4 
Manufacturing 17 
Transportation 4 
Business Services 5 
Health Care 7 
Education 

Multiple Industries 7 


0 1 3 
2 12 3 

0 2 2 

1 2 2 
1 2 4 

0 2 2 

1 4 2 



* Industries with fewer than four responses omitted from analysis. 

Adds to more than 27 because projects could select more than one industry. 
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Table F-7 

RANKING OF PROJECT GOALS 





Project Goals 




Increase 


Establ Ish 


Improve 




Project Name 


Access 


Partner 


Training 


other 


Alabama Aviation College 


c 




1 




Ben Hill-Irwin Institute 




2 


1 




Bronx Community College 


I 


Z 






CORD 


Z 




1 




Clackamas Community College 


z 


1 






Columbia Basin College 




L 


1 
1 




Fox Valley Tech 




0 
L 


1 
1 




Hampden County Consortium 


1 
I 




c 




Home ouiiuers institute 


o 

L. 




1 
1 




nowaru Lommunity ldi lege 


L 




1 
1 




Illinois tastern Loiiege 




C 


1 
X 




maian hi lis Loiiege 


1 


L 






oonn n. ratterson uoiiege 




O 
L 


X 




Liv oteei Lo. 




2 




1 


Luzerne Lommunity uoiiege 


1 




2 




Nebraska Labor Department 


2 


1 






North Clackamas Schools 




2 


1 




Northampton Community College 




2 


1 




PAVE 




1 




2 


State Center College District 


2 




1 




Valencia Community College (Health) 




2 


1 




Valencia Community College (Tele) 


1 


2 






Valencia. Community College (CIM) 


1 


2 






Waubonsee Community College 


1 




2 




West Virginia Northern College 




2 


1 




West Virginia Department of 




2 


\ 




Education 1 










West Virginia Department of 




2 


1 




Education 2 
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The distinction between the projects with a training focus and 
those with other emphases appeared to persist as projects selected 
second ranked goals. Of the 12 grantees that did not select training 
as the first goal, only three selected it as second. Grantees that did 
select training as their primary goal selected the goal of increased 
access to vocational training for special populations somewhat less as 
a second goal than did other grantees* Of the 15 that selected 
training first, only five selected increased access for special 
populations as their second goal. This group was more likely to choose 
public/private partnerships second, with 10 making that selection. 
What these choices suggest is that increased access of special 
populations was the primary or secondary goal of a distinct subset of 
projects— those that saw the goal of developing new or improved 
training as somewhat less important.^ 

Projects that selected new or improved training as their main goal 
also differed from those that selected increased access in what they 
considered to be their "key" project activities. Projects that held 
new or improved training as their most important goal were more likely 
to view curriculum development as their key project activity (see Table 
F-8). Of the training projects, five of 15 saw curriculum development 
as the most important activity. In contrast, among the projects 
emphasizing increased access of special populations, none saw 
curriculum development as its most important project activity. The 
"increased access" programs were more likely to emphasize the training 
itself as the most important activity— with four of seven making this 
selection. 

As for the meaning of high technology, the projects also reflected 
different interpretations of the Congressional mandate. Based on case 
studies of FY 1988 grantees, the study team identified four basic ways 
in which projects incorporated high technology in developing their 
programs and delivering training. Projects were high-tech if: 

■ training was designed to prepare students for 
jobs in fields that manufacture high-tsch 
products or service high-tech equipment; 
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■ training was designed to enable students to 
use high-tech equipment or products even 
though the field in which the equipment is 
used is not generally considered high tech; 

■ training was conducted on high-tech 
equipment, such as computers, CAD, or CIM 
equipment; or 

■ training was offered in basic skills as 
preparation for specific occupational 
training in a high-tech field. 

In the survey, FY 1989 grantees were asked to select all the high-tech 
definitions appropriate to their projects. 

Most grantees selected more than one response to describe their 
projects' high-tech elements (see questionnaire, Item 13). Fourteen 
projects selected [a], 12 selected [b], 17 selected [c], and 13 
selected [d]. To make it possible to analyze these data further, the 
team observed the overlap between [a] and [b], which asked respondents 
to select essentially mutually exclusive fields of employment. The 
study team discovered that there were only three projects that selected 
both responses, and that a total of 23 projects had selected [a], [b], 
or [a] and [b]. Of the remaining four projects, one selected [c], two 
selected [d], and one selected [c] and [d]. The study team reasoned 
that projects selecting [a] or [b] and also making selections from 
among [c] or [d] were. In making their [c] and [d] selections, picking 
activities that occur as a function of training provided in fields 
described in [a] or [b]. Hence, for these projects the team would 
concentrate further analysis on the field selection— 1 .e. , [a] or [b]. 
As a result, the team created one response category for each project 
for further analysis as follows: 

■ [a], but not [b]: training for high tech 
field— 11 respondents; 

■ [b], but not [a]: training for high-tech 
equipment In non-high-tech field— nine 
respondents; 
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■ [a] and [b]: training for both high-tech 
field and for using high-tech equipment in a 
non-high-tech field— three respondents; 

■ [c] only: training on high-tech 
equipment— one respondent; 

■ [d] only: basic skills training to prepare 
for further training in a high-tech 
field— two respondents; and 

■ [c] and [d] only: training on high-tech 
equipment and basic skills training in 
preparation for further training in a high- 
tech field— one respondent/ 

Using these categories, most of the projects (23) defined "high- 
tech" in terms of the occupation or equipment for which training was 
provided. Of those projects, about half (the 11 [a]s) were explicitly 
preparing persons for immediate or specific work in high tech fields* 
Most of the others (the nine [b]s) were preparing students to use high- 
tech equipment or products in a non-high-tech field, with the rest 
doing both. If these answers are indicative, the remaining four 
projects were not preparing students for explicit high-tech 
applications, but rather defined their projects' high-tech element in 
terms of the equipment used in training or the students' long-range 
occupational goal. 

Second, the study team examined more closely the extent to which 
the 23 projects that defined their high-tech focus in terms of 
occupation or field application also used high-tech equipment or 
provided basic skills instruction as part of their activity (see Table 
F-9). The team found that of the programs that prepared students for 
work in high-tech fields, the majority (eight of 11 under [a] above) 
used high-tech equipment in training. Among the projects preparing 
students to use high-tech equipment in non-high-tech fields, however, 
only a slim majority (five of nine under [b]) conducted training on 
high-tech equipment. Because these projects were explicitly preparing 
students to use high-tech equipment or products on the job, the lack of 
training on such equipment in a majority of them is surprising. It 
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Table F-9 

EXTENT TO WHICH PROJECTS THAT PREPARED STUDENTS TO WORK 
IN HIGH TECH FIELDS OR WITH HIGH TECH EQUIPMENT TRAINED 
STUDENTS ON HIGH TECH EQUIPMENT OR IN BASIC SKILLS 





Trained on High 
Tech Eauipment 


Target Basic Skills 


High Tech Definition 
of Project 


Yes No 


Yes No 


a* Prepared Students for Jobs 
in Fields Manufacturing or 
Servicing High Tech 


8 3 


6 5 


b. Prepared Students to Use 
High Tech Products 
Although Field Was Not 
High Tech. {n=9) 


5 4 


2 7 


a. and b. (n=3) 


2 1 


2 1 
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suggests that the training may have been quite limited in scope. 

The projects preparing students to use high-tech equipment in non- 
high-tech settings were also more likely not to offer basic skills 
instruction. Only two of nine such projects indicated basic skills 
instruction, compared with six of the 11 projects preparing students 
for jobs in high tech fields. This finding also suggests that the 
training for use of high-tech equipment in non-high-tech fields may 
have been rather limited or narrow in scope. 

Projects that included manufacturing were considerably more likely 
than others to be preparing students for work in high-tech fields (see 
Table F-10). Eleven of 17 projects that indicated preparation for the 
manufacturing industry— 65 percent— selected either [a] or [a] and [b] 
combined. Projects described as training for multiple industries also 
were weighted to preparation for high-tech fields, with four of seven 
selecting this choice. This compares with only one of seven projects 
that provided preparation for health care jobs, with five of the seven 
respondents selecting [b]. None of the four projects that included 
training for the transportation industry provided training for a high- 
tech field, and business service projects were about equally divided 
between high-tech and non-high-tech fields. 

Project Clientele: Overview 

One of the important goals of the Perkins Act has been to increase 
the access of special populations to vocational training. While the 
specific mandate of the Cooperative Demonstration Program with respect 
to special populations is ambiguous, and the grants announcement is 
silent on the issue, the importance of the goal to the overall Act 
makes it worthy of examination. The questionnaire included a number of 
questions designed to describe the special populations that were served 
through Cooperative Demonstration projects. 

As noted previously, a subset of Cooperative Demonstration 
projects considered increased access of special populations as their 
most important goal. Further examination shows that these projects 
were also more likely to report that their project served a clientele 
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that was new or different for the institution (see Table F-11), When 
asked whether the project benefitted a new or similar group of clients, 
three of the seven projects that selected the increased access goal as 
primary also indicated that the project clientele was new or different. 
The other four projects with the access goal indicated that they served 
new groups as well as groups similar to those served in the past. None 
of these programs indicated that its project served a clientele 
entirely similar to groups served previously. By contrast, only one of 
the 15 projects that selected the improvement of training as its first 
goal also indicated that it served a new group exclusively, and seven 
indicated that they served groups that were entirely similar to those 
served in the past. An additional seven programs indicated that they 
served both new groups and groups similar to those served in the past. 

The projects that held increased access as their primary goal were 
also more likely to report that they serve disadvantaged persons (see 
Table F-12), Comparisons of project goals with responses to the open- 
ended question on project clientele indicated that of the seven 
projects with increased access as the main goal, three characterized 
their primary clientele as adults seeking work in a particular field or 
company, three characterized their clientele as disadvantaged persons, 
and one indicated that it served existing employees of companies. None 
of the programs that saw new or improved training as their primary goal 
indicated that they served disadvantaged persons primarily, with seven 
of 15 indicating current employees as their primary clients and the 
majority of the rest indicating adults seeking work. Or to put it 
another way, three of the four programs serving disadvantaged persons 
as their primary clients also indicated that increased access was their 
main goal. Seven of the 13 projects serving current employees of 
companies indicated that new or improved training was their main goal. 

Despite the rather limited number of projects that indicated they 
served special populations primarily, the majority of projects reported 
making specific efforts to recruit special populations (see 
questionnaire, item 24), All 19 projects that conducted any 
recruitment activities also reported such efforts, with the largest 
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Table F-11 

PROJECTS' PRIMARY GOAL AND LIKELIHOOD OF SERVING 
NEW OR DIFFERENT CLIENTELES 





Clientele Benefitting From Pro.iect 


Primary Goal of Project 


New or Different 
Group for 
Institution 


Similar to Past 
Groups Served 
by Institution 


Both 


Increased Access for 
Special Populations 
(n«7) 


3 


A 

V 


A 
H 


Create New or Improved 
Training (n«15) 


1 


7 


7 


Establish Partnerships 
(n=3) 


0 


1 


2 


Economic Development 
(n«2) 


1 


0 


1 
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number trying to recruit women and blacks (18 projects). Hispanics and 
unemployed persons were a focus of recruitment efforts in 16 projects, 
followed by low-income persons (12 projects), and persons with 
disabilities (11 projects). Other populations targeted by projects 
with less frequency included Asian or Pacific Islanders American 
Indians or Alaskan Natives, persons reading below the eighth-grade 
level, immigrants, and persons living in rural areas. The methods used 
to recruit special populations varied, but distribution of posters, 
flyers, or other materials, advertisement in print or broadcast media, 
internal recruitment within the grantee organization/district, and 
contact with government offices (including PICs and employment 
services), were among the most popular. 

Although most projects made some efforts, the four projects that 
indicated they served disadvantaged students were more likely to have 
made extra efforts to recruit these groups (see Table F-13). 
Interestingly, only four of the seven projects that served adults 
seeking work made extra efforts to attract special populations, 
suggesting that they may already have had a readily available source of ^ 
such students. The three projects that served high school students 
also made efforts to attract special populations.® 

As noted previously, business services projects were somewhat more 
likely to indicate that they served a new group than were projects 
training in manufacturing or health. It is hard to reach any 
conclusions about the relationship between primary clientele and 
industry of training, however, because only four projects identified 
disadvantaged persons as their primary clientele. If the definition of 
"disadvantaged" is broadened to include adults seeking work in a field 
or company, however, a pattern of relationships between industry and 
clientele served begins to emerge. 

If projects serving disadvantaged persons and adults seeking work 
are combined, the survey shows that projects in areas other than 
manufacturing or "multiple industries" were more likely to serve these 
populations (see Table F-4). Three of five projects providing training 
in business services, three of four projects with training for 
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Table F-13 

PROJECTS' PRIMARY CLIENTELE AND LIKELIHOOD OF SPECIAL- 
POPULATION RECRUITMENT EFFORTS 
(NUMBER OF PROJECTS) 



Primary Clientele* 


Soecial Pooulation Recruitment Effort 
Yes No 


Current Employees of Companies 

Adults Seeking Work in Field or 
Company 

Disadvantaged Persons 
High School Students 


8 .5 
4 3 

4 0 

3 0 



* Categories derived from open ended question. 
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transportation, and five of seven offering health care training 
primarily served these groups (combined) • On the other hand, only six 
of 17 projects providing training for manufacturing and three of seven 
that trained in multiple industries served these groups primarily* 

The good news, then, is that the projects most likely to serve 
special populations were also those conducting training in fields less 
affected by the economic downturn* Whether this occurred by design or 
by default is unknown, however* Was it the initial plan of the 
manufacturing training projects to focus on current employees, or did 
projects offering this training adapt to a poor job market by serving 
those already employed? Did projects offering training in health care 
or business decide to focus on persons seeking work or disadvantaged 
persons because they believed jobs were available? 

Projects serving adults seeking work and disadvantaged persons 
also appeared somewhat more liicely to provide training aimed at the 
development of skills for entry-level work (see Table F-14). Three of 
the 11 projects (27 percent) aimed at the two groups combined reported 
providing entry-level skills training exclusively, with the eight 
remaining projects indicating that they provided training for both 
entry-level and upgraded skills. None of these projects reported 
providing instruction aimed exclusively at upgrading skills. Of the 13 
projects serving current employees, three (23 percent) indicated that 
they provided entry-level skills exclusively, but four (31 percent) 
indicated that they provided upgrade skills exclusively. The six other 
projects indicated that they provided both. 

Given the finding that projects serving adults seeking employment 
and disadvantaged persons tended to offer training in fields other than 
manufacturing, and the earlier finding that manufacturing projects were 
more likely to prepare students for work in high tech fields, it is not 
surprising to find that disadvantaged persons and adults seeking 
employment also were less likely to train for high-tech fields (see 
Table F-15). Of the 11 projects that emphasized these populations, 
seven were preparing students to use high-tech equipment or products in 
fields not generally considered high-tech. Only three were preparing 



ERLC 



330 



F-28 



Table F-14 

PRIMARY CLIENTELE AND SKILL LEVELS OF TRAINING 
(NUMBER OF PROJECTS) 



Primary * 
Clientele 


Entry-level 
Skills 


Upgrade 
Skills 


Both 


Current Employees of 
Companies 


3 


4 


6 


Adults Seeking Work in 
Field 


1 


0 


6 


Disadvantaged Persons 


2 


0 


2 


High School Students 


2 


1 


0 



* Categories derived from open ended questions. 
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students to work in high-tech fields, and one indicated that it was 
using high-tech equipment in training and providing training in basic 
skills. In contrast nine of 13 projects for current employees were 
training students for work in high tech fields or a combination of 
high-tech and non-high-tech fields. 

Projects aimed at disadvantaged persons and adults seeking work 
also were more likely to provide non-occupation specific training in 
addition to occupation specific training (see Table F-16), All four 
projects for disadvantaged persons provided non-occupation specific 
training as did five of the seven projects for adults seeking work. In 
contrast, only five of the 13 projects for current employees provided 
such training. This finding may indicate that the current employees 
were considered by project staff to have sufficient background to 
pursue occupational training without additional assistance, or that the 
training provided was relatively narrow in scope and, hence, required 
little supplementary instruction. 

Pro.iect Clientele: Information Drawn from Participant Counts 

In addition to characterizing their overall clientele, projects 
were asked specific information about participants. These included 
participants* age, race, sex, proficiency in English, special 
population status, level of education attained, primary educational or 
vocational goal, employment status at the time of training, training 
status at the end of the grant period, and employment outcomes. 
Unfortunately, fewer than half the projects were able to supply 
information beyond total number of participants, sex, race, and 
employment status during training. The projects that failed to supply 
participant information also were those that claimed the largest 
numbers of participants. Nonetheless, some findings can be reported.. 

The total number of participants varied widely by site (see 
questionnaire, item 22). Twenty-six projects provided head count 
enrollments (not necessarily unduplicated) that ranged from nine to 
1,589, with a median enrollment of 113 students. Only 14 projects also 
were able to supply FTE enrollment figures, enrollments that ranged 
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Table F-16 

PROJECTS' PRIMARY CLIENTELE AND LIKELIHOOD OF PROVIDING 
NON OCCUPATION-SPECIFIC INSTRUCTION 



Primary Clientele* 


Non-Occuoationallv Specific Instruction 
Yes No 


Current Employees of Companies 

Adults Seeking Work in Field 
or Company 

Disadvantaged Persons 
High School Students 


5 8 
5 2 

4 0 

2 1 



* Categories derived from open ended question. 
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from 19 to 517, with a median of 30 students. 

Males were the largest number of training recipients (see Table F- 
17, and questionnaire table 1). For the 79 percent of participants for 
whom sex data were provided, 67 percent were male, and 33 percent were 
female. Further, males were concentrated in the programs providing 
training in manufacturing, while females were concentrated in the 
programs providing training in business services and health care (see 
Table F-18). Although 19 programs indicated that they made special 
efforts to attract women, it would not appear that the projects 
provided a testing ground for nontraditional training by sex. 

For the 21 projects reporting the race of most or all trainees, 
the majority were white (see Table F-17). In these projects, whites 
constituted approximately 83 percent of participants for whom race was 
reported, blacks were 11 percent, Hispanics were five percent, Asians 
were one percent, and American Indians/Alaskan Natives were less than 
one percent. Fewer than two percent of participants were limited 
English proficient. Overall, whites constituted an even greater 
percentage of participants in these projects than represented in the 
population. So, although 19 grantees said that they made special 
efforts to recruit minorities, among the 21 projects that supplied 
information, the program appears to have done little to demonstrate new 
opportunities for minorities. The only caveat is that the seven 
projects with missing data on race include about half the 
participants.* 

The majority of trainees were employed full time during training. 
Of the 87 percent of participants for whom employment status was 
reported, 85 percent were employed full time during the period of 
training (see questionnaire table 1). Further, most of these persons 
were employed by the private-sector partners that cooperated in the 
projects. Of the 80 percent of full-time employees for whom the 
employer was known, 92 percent were employed by a partner company. The 
total percentage employed by partners was probably somewhat less than 
this figure, of course, because employment by the partner was more 
likely to be known to the grantee. Nonetheless, one can safely 
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conclude that the main training recipients were full-time employees of 
partner organizations. 

Understanding the employment status of the clientele makes it 
easier to explain the responses to a question that asked grantees to 
indicate the number of participants seeking various educational or 
vocational outcomes from the training (see questionnaire table 1). 
Choices provided to respondents included job placement, job promotion, 
vocational certificate, high school diploma or GED, college credit, 
associate degree, and "other." The most popular educational outcomes 
for participants reported by grantees was a write-in category that 
indicated 36 percent of students sought specific job skills. The 
second most common educational goal was "unknown": 19 percent of 
students; followed by job promotion: 17 percent of students; and job 
placement: 14 percent of students. Projects reported that nine percent 
of project clients sought college or CEU credits. Either most students 
did not seek credentials or programs did not provide them and, hence, 
did not know whether students were seeking them. Only six percent of 
students were reported as seeking any credential (including a high 
school diploma or GED, vocational certificate, or associate degree). 
Combined with the findings that most participants were employed, and 
that mastering a set of occupational skills was one of the most popular 
likely "rewards" for students from the projects' perspective, these 
findings suggest that projects were providing forms of training 
tailored to very specific job requirements. 

Pro.iect Activities 

In order to understand how projects operated, the questionnaire 
asked respondents to identify the main project activities and the 
amounts of staff time devoted to each. Grantees were asked about 
activities conducted as part of project development as well as direct 
services to clients. The responses to these questions showed the 
tremendous diversity of emphases and approaches undertaken through the 
demonstrations. The variety of responses also points out the problems 
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inherent in attempting to compare implementation across such disparate 
interventions. 

Preparatory Activities , There are great ranges in relative 
emphasis in the activities undertaken by projects in preparation for 
offering services to clients. The most common preparatory activity was 
to identify the skills or training needed in a particular field or 
geographic area (see questionnaire, item 18). Twenty-two projects 
indicated that they carried out this activity, but the staff hours 
devoted to the activity ranged from 10 to 2,234. Across all projects, 
the mean number of hours was 272 and the median was 175. Other time- 
consuming start-up activities included: 

■ Recruiting and hiring staff (21 projects, 
median hours: 50); 

■ Development of student assessment materials 
(20 projects, median hours: 65); 

■ Recruiting students (19 projects, median 
hours: 100); 

■ Review of planned offerings by experts, 
potential employers or institutional 
officials (16 projects, median hours: 50); 

■ Recruiting employers interested in having 
employees trained or hiring students (16 
projects, median hours: 85); and 

■ Contracting for training (15 projects, median 
hours: 50). 

Some teacher training was undertaken in most of the projects, but 
the amount of training varied widely (see questionnaire, items 38 and 
39). Sixteen projects indicated that they provided instructors or 
other staff with preservice or in-service training in order to teach in 
the project. An additional eight projects reported that such training 
was not necessary. Two projects did not provide training but, in 
hindsight, thought that it would have been valuable, Preservice 
training for instructors occurred in 12 projects and varied from six to 
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763 hours per project (number of staff trained times number of 
instructional hours), with a mean of 158 hours but a median of only 60 
hours. In-service training occurred in 11 projects and ranged from 10 
to 2,000 hours, with a mean of 357 and a median of 200 hours. What 
this suggests is that a few projecls spent a great deal of time on 
preservice and in-^service training, while the majority spent little. 

Curriculum Development . As noted previously, five projects 
considered curriculum development to be their most important activity, 
but every project reported that it spent some time selecting a 
curriculum (see questionnaire, item 34). Ten projects reported that 
the curriculum was developed entirely by project staff, while nine said 
their curriculum was adapted or adopted from a curriculum already in 
use within the grantee institution. One project indicated that it 
adopted or adapted a curriculum in use at another institution, and one 
other adapted/adopted a curriculum already used in industry. Six 
projects used a combination of approaches. Given that only one project 
used a curriculum adopted/adapted from another institution, it would 
appear that replicating an approach tried elsewhere was not a major 
emphasis in this demonstration program. 

Overall, projects devoted considerable time and effort to 
curriculum development (see questionnaire, item 18). Staff hours 
ranged from eight to 3,000, with an average of 539 but a considerably 
lower median number of 241 hours. Twenty-two projects indicated that, 
as part of curriculum development, they worked with private industry to 
identify training needs and skillrequirements. Most also worked with 
industry to design the curriculum (21 projects), gain approval (19 
projects), and pretest and revise the curriculum (16 projects). Five 
projects also consulted with union officials in developing curriculum. 

As part of program development, almost all of the projects (24) 
sought to identify the specific job skills or qualities sought by 
employers (see questionnaire, item 33). To identify those skills the 
projects interviewed practitioners in the field (21 projects), surveyed 
employers (20 project^:), reviewed standard or existing skill- 
requirement information (19 projects), observed skills at work sites 
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(16 projects), reviewed employer job descriptions (16 projects), and 
conferred with experts (15 projects)* In addition, nine projects 
reviewed previous reports and/or studies in an effort to identify the 
job skills or qualities employers sought. 

The primary goal of the project did not appear to be a major 
determinant of whether the curriculum was developed anew or 
adapted/adopted (see Table F-19). Projects having a primary goal of 
improved training were somewhat more likely to develop new curricula 
than projects primarily concerned with increased access, but the 
differences were not large. 

Training . In general, grant-supported training was of short 
duration (see questionnaire table 1). Grantees reported that 68 
percent of participants received between one and 100 hours, while 11 
percent received 101 and 250 hours (see Table F-20). Approximately 14 
percent of participants received between 251 and 1,000 hours of 
training and 14 pe*^cent received 1,000 or more hours of training. 
There is a major caveat with respect to the hour totals, however— one 
that suggests that the actual number of grant-supported hours of 
training may be lower than these numbers reflect. Not all projects 
interpreted total instructional hours received through the grant in the 
same manner. The questionnaire instructions asked for "hours of 
instruction. . .from the project during the grant period," but it is 
likely that at least some projects included total instructional hours 
participants received, whether or not the training was project 
supported. For example, a project might have provided revised or 
additional services to an ongoing course or degree program. Some sites 
reported the total number of hours required to complete that course or 
degree program as the instructional hours received under the project. 
Other projects— especially those where grants largely supported 
separate training— reported only the portion of the course or program 
hours affected by a grant-supported activity or curriculum. 

Attempts to obtain data on the methods of instructional delivery 
yielded few responses, and these reflected a wide variety of 
approaches. Open-ended questions that asked about the type of 
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instruction students received (e.g., lecture, lab) and other services 
provided (e.g., counseling) were answered by only 13 and 11 projects, 
respectively, and the responses varied widely in terms of service modes 
(see questionnaire, item called table 1). Among the responding 
projects, applied learning was quite common, with some combination of 
laboratory and lecture formats popular. 

In addition to occupation-specific training, 16 projects included 
non-occupation specific instruction or training as part of their 
program (see questionnaire, item 31). The non-occupation-specific 
skills taught most often included employability skills (13 projects), 
basic or remedial academic skills (11 projects), or advanced academic 
skills (nine projects). 

Student Assessment . All but two projects reported that they 
conducted some type of student assessment, but few conducted the same 
assessments at entrance and later in the training (see questionnaire, 
item 32). The most common method was assessment of job skills, with 20 
projects reporting this method. Of those 20 projects, 12 assessed job 
skills at entrance, and 17 assessed job skills once instruction had 
begun. Nine projects conducted job skill assessments both at entrance 
and during training. The second most common form of assessment was the 
administration of an academic aptitude test, which 15 projects gave at 
entrance. Only three of the 15 projects administered this form of test 
again after training had begun. 

In fact, beyond job skill, the assessments favored at entrance and 
during training were different. As noted, the use of academic aptitude 
tests dropped off after entrance. In contrast, the use of criterion- 
referenced achievement tests increased. Seven projects administered 
criterion-referenced tests at entrance, but 11 administered them once 
training had begun. Only five projects administered criterion- 
referenced tests both at entrance and during training, however. 
Vocational aptitude tests were used by nine projects at admission, but 
only three projects thereafter. Eight projects administered a 
standardized achievement test at entrance, but only one project 
administered such a test thereafter. The general lack of consistency 
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between baseline and subsequent student assessments suggests that, with 
the exception of nine projects that conducted baseline and follow-up 
assessments of job skills, projects would be unable to determine their 
impact on students' performance. Given that these projects were 
intended to be demonstrations, the built-in inability to provide 
evidence of effectiveness is most disheartening. 

Support Services . Based on the findings from the first year of 
the evaluation, the study team anticipated that projects would offer 
participants services in addition to training. To determine the range 
of services, projects were asked about the types of services they 
offered directly, as well as those provided by another agency in 
coordination with the project (see questionnaire, item 20). Overall, 
21 projects indicated that they provided support services directly or 
coordinated with another agency to provide the service. 

Aside from student assessment and academic remediation, which have 
been discussed already, the most common services were those that 
involved career information. Employability advice was available in 20 
projects— including 13 that provided it directly and through 
coordination, as well as 10 that provided it entirely through 
coordination. Also commonly available were career or other counseling, 
which 17 projects provided. Of those, 15 provided it directly, and two 
provided it in coordination with another agency. Three projects 
offered both. Job placement services were available in 16 projects, 
with 13 providing the service directly. The only other service 
available in more than half of the projects was tutoring, which was 
available in 15 projects and directly provided by 12. The nature of 
the support services overall appears to reflect the fairly job-specific 
focus of the projects. 

Given the rather small amount of training per student and a 
clientele that was for the most part employed full time, a substantial 
minority of projects offered students financial assistance. Eight 
projects indicated that they provided stipends or other financial aid 
directly to students. Only one of these projects reported that 
students were eligible for financial aid through another source. Six 
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projects said they provided students with transportation assistance 
directly, and one coordinated assistance. Four projects provided child 
care directly, and five projects offered health care directly or 
referral for health care. 

Public/Private Cooperation 

The Cooperative Demonstration Program was designed to encourage 
cooperation between educational institutions and the private sector. 
As the grants announcement noted: 

High technology training can be conducted most 
effectively with the active involvement and 
cooperation of the private sector. Effective 
partnerships between the private sector and public 
agencies in vocational education are an important 
aspect of the Cooperative Demonstration Program 

The announcement anticipated that the partnerships established by the 
project would provide models of effective cooperation. To understand 
the nature and extent of cooperation, the survey asked numerous 
questions about the projects' public/private partnerships. 

Most, but not all, projects established relationships with 
employers (see questionnaire, item 40). Of the 27 respondents, 23 
indicated some involvement of employers in planning or administering 
the project or in providing services. Of the remaining four projects, 
only one indicated that no outside partnership was formed. Some 
projects had multiple types of partners including schools or 
universities (institutions that might be employers or may be involved 
in the project in some other manner-~19 projects) and community-based 
organizations (10 projects). 

Although employers were involved in 23 projects, not all of these 
projects indicated that private businesses were the most important 
partner organizations (see questionnaire, item 42). Slightly more than 
half the projects (14 of 26) indicated that the most important partner 
was a private business. Of the remaining projects, five said that the 
most importf.nt partner was a trade association or consortium, four 
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indicated an educational institution, and three indicated a non- 
educational public agency. It would appear that, in at least the five 
projects linked with an association or consortium, the most important 
private-sector partner was not an institution capable of employing 
persons trained by the program. For grantees that had second partners 
(see questionnaire, item 51), these partners were even less likely to 
be private businesses (nine of 20) and more likely to be trade 
associations (four of 20). 

Although most grantees formed partnerships with only a few 
organizations, some established multiple partnerships (see 
questionnaire, item 41). Six projects had one private sector partner, 
and 16 projects had four or fewer, but seven projects indicated that 
they had 10 or more partners. Given the tremendous range in the number 
of partners, it is likely that the roles of partners differed 
considerably across the projects. 

In fact, differences in partner contributions were considerable. 
When asked to rank in importance a list of possible activities for the 
most important partner organization, there was little consensus among 
the grantees (see questionnaire, item 44). The most commonly selected 
first choice activity fjr the partner organization was to provide 
equipment for training, but that choice was selected by only seven 
grantees. Five grantees said that the partner's main contribution was 
identifying job skills, while four indicated that the partner recruited 
students for training. Three indicated that the partner served on a 
project advisory committee, and two indicated that the partner supplied 
instructors. In short, there was considerable variability in first 
partner contribution— a variability that is sustained when the first- 
through third-ranked activities are combined. The same findings hold 
true for second partners, and, in fact, the variability in contribution 
increases (see questionnaire, item 52). 

The differences in activities may be, in part, a function of the 
nature of the partner relationships. Based on categories developed 
from site visits to FY 1988 grantees, the study team identified several 
ways to characterize or summarize the relationships between grantees 
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and employers* Each FY 1989 grantee was asked to pick the choice that 
best characterized its relationship with its first partner (see 
questionnaire, item 45). The majority of grantees selected one of two 
choices: 

■ The partner was a customer of the project, 
e.g., the project provided customized 
training to the partners' employees (selected 
by nine grantees); or 

■ The partner participated in the delivery of 
instruction and services (selected by nine 
grantees) 

No other choice was selected by more than three grantees (see Table F- 
21). The nature of the relationship was then matched with the most 
important partner activity. 

In most of the projects, the grant did not result in the 
establishment of a new primary partnership (see questionnaire, item 
46). Nineteen of 27 grantees reported that the relationship between 
the grantee institution and the partner organization was not new, but 
all 19 also said that the relationship had been strengthened as a 
result of the project. The eight projects that did establish new 
relationships with the first partner organization were more likely to 
have encountered problems. Three of six projects that reported any 
problems with their first partner had new partners, and two of these 
partnerships were dissolved at the end of the grant. 

Grantees were slightly more likely to have established new 
partnerships with their second partners (see Questionnaire, item 54). 
Eight of 21 second partners were new, and appeared to reflect a wider 
range of relationships. Considerably fewer second partners (2) were 
project customers (i.e., they purchased customized training or hired 
students), though a sizeable number (eight) shared in delivery of 
instruction and services. This difference between first and second 
partners may have occurred because second partners were more likely to 
be trade associations or other educational institutions rather than 
employers. Second partners were somewhat more likely to serve 
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primarily as members of advisory committees. 

Grant Operations 

With any Federal demonstration program, there is interest in 
whether funds provide an opportunity that would not otherwise have been 
available and do not simply substitute for existing resources^ It also 
is hoped that Federal funds will provide the impetus for funding from 
other sources so that activities can be expanded or continued after the 
Federal support has ended. To address these policy issues, respondents 
were asked about the importance of Federal funds in project startup and 
development, as well as the likelihood of project continuation beyond 
the grant period. 

In addition, based on the examination of the FY 1989 grantees, 
there was a specific concern about the ability of grantees to start up 
quickly and to complete their projects. In general, the lengthy period 
between proposal submission and award notification, the mid-academic- 
year starting date for most grants, and the 18-month duration of the 
grant all contributed to startup and completion problems for some of 
the grantees. For example, when they submitted their proposals, 
several projects had identified persons to be hired, but those persons 
had taken other jobs by the time the awards were made. Finding new 
staff who were willing to work on a closed-end, 18-month grant took 
considerable time, and several FY 1988 projects never had permanent 
project directors. In addition, the mid-academic-year startup date 
made creating new training classes impossible to arrange until the 
following fall semester, so up to 10 months elapsed before training was 
offered. To begin to determine the extent to which these problems 
persisted among FY 1989 grantees, the survey asked whether the projects 
were able to start to provide services to clients immediately upon 
notification of awards and if not, how long thereafter. 

Most FY 1989 grantees said their projects would not have existed 
without Federal support (see questionnaire, item 17). Of the 27 
projects, only four respondents indicated that the project would have 
existed without the Federal funds. Three of those were projects 
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providing the services prior to the start of the grant* At the 
completion of the grant period, 14 projects expected to continue in 
their entirety, with another eight indicating that they would continue 
in a scaled-down form* Two projects reported that their Federal 
funding had not yet lapsed. 

Projects generally began to provide direct services to clients 
within a few months after their grants began, but a few did not provide 
services for many months (see questionnaire, item 14). Excluding the 
three projects that were providing services to clients before the grant 
began, 12 projects began to provide services within three months of 
being awarded the grant, and nine more began within nine months. Three 
projects did not start, however, until 10 to 12 months after the grants 
were awarded (or six to eight months before the grants were originally 
due to end) . 

The grant period nonetheless appears to have been sufficient for 
many projects to have become institutionalized or to find other sources 
of support (see questionnaire, items 15 and 16). Twenty-two projects 
indicated they continued beyond the end of the grant. The most common 
sources of support among continuing projects were funds from the 
grantee institution (11 respondents), with private employers the second 
most common (eight respondents had commitments from employers and four 
were waiting for final agreements). Five respondents planned to charge 
students tuition for project services. 

The ability of projects to provide services soon after receiving 
an award appeared to have little or no effect on whether they continued 
services beyond the grant period (see Table F-22). In fact, there was 
a slightly greater tendency for projects that began providing services 
last to continue, with all of the latest-starting programs continuing 
in their entirety after the completion of the grant. Two of the three 
projects that indicated they were providing services before the grant 
started either received extensions to December 1991, or were no longer 
providing services at the time of the survey. 

Yet when asked directly about factors affecting their ability to 
complete their original plans, the short grant period was noted by the 
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largest number of respondents (see questionnaire, items 59-61), Nine 
projects indicated th^y had encountered problems in completing their 
original plans. The most commonly cited reason for implementation 
problems was that the 18-month grant period was insufficient, a reason 
cited by five of these nine projects. Also noted by more than one 
project were difficulties in staff recruitment and/or retention (two 
projects) and planned activities that proved inappropriate (two 
projects) . 

At the same time, however, 10 projects indicated that they had 
accomplished activities they had not originally planned (see 
questionnaire, items 62 and 63). The most commonly cited additional 
activity was curriculum development (five projects), followed by 
partner recruitment (four projects), and dissemination (three 
projects). Over half of these projects (six) indicated that they 
obtained additional funds that enabled them to undertake these 
unplanned activities. 

The ExDortabilitv of Project Activities 

Because this was a demonstration program, respondents were asked 
what features of their projects had applicability for or could provide 
useful models for other educational or employment training programs. 
Respondents were provided with a list of possible project features and 
asked to rank up to three they considered applicable or useful for 
others. 

The projects were quite divided in choosing features of their 
programs that provided information for others (see questionnaire, item 
12). The most commonly selected first choice was customized training 
for a particular employer or group of employers, which was selected by 
six respondents. The second most commonly selected feature, selected 
by five respondents, was a new or improved curriculum. "New or 
improved kind of training," "established or strengthened public/private 
partnerships" and "applied high technology equipment to the delivery of 
training" each received four votes. Three projects chose "expanded 
access to training for disadvantaged or under represented groups," 
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Only one selected •'model of school -to-work transition," which was one 
of the few demonstration objectives specifically identified in the 
grants announcement* 

This wide range of responses can be read several ways. Given that 
this was intended to be a demonstration of public/private partnership 
for training in high-tech fields, the lack of consensus on exportable 
features is surprising. One might have expected the partnership or 
training choices to account for a large number of first choices, but 
thv"iy account for only eight of 27. The selection of customized 
training as a first choice is surprising, because, by its nature, 
customized training is rarely applicable across industries or 
educational institutions. 

If all three choices are added together, however, establishing or 
strengthening public/private partnerships does appear to be the featu e 
most commonly selected as having widest applicability. Twenty projects 
considered partnership among the top three exportable features. Con- 
versely, few projects that did not select customized training as a 
first choice selected it second or third, making it one of the lowest 
ranked choices overall. Partnerships was the only exportable feature 
identified by more than half the projects, even with first through 
third choices are added together. 

The lack of consensus on what was being "demonstrated" is 
troubling. While it is not possible to reach conclusions from the 
responses to one question, it does seem fair to say that grantees did 
not share a common view of what they had used Federal support to "try 
out." And although all projects were, according to the grants 
announcement, supposed to "try out" public-private cooperation for 
high-tech training, only 20 projects considered their public/partner- 
ships, and only 13 projects considered their training as applicable to 
or providing a useful model for others. Nonetheless, it should be 
noted that 25 projects undertook some activities to disseminate the 
curricula they used. 
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Notes 



I. Two projects claimed that extensions to their projects made it impossible to 
provide needed data, but the evaluation staff indicated that final tallies of 
students or staff were not required. Some additional information was obtained 
from one nonresponding site at interviews conducted during a subsequent site 
visit. In order to maintain the integrity of the survey component of the 
analysis, however, the data summarized here are information collected from sites 
that returned project questionnaires prior to site visits and do not include 
information collected during the visits. 

Z. Respondents were allowed to choose as many responses as were appropriate to 
their project. 

3. It is likely that these five were projects with more than one kind of training 
provided. 

4. The issue of industry and special populations is discussed in greater detail 
under the special populations heading. 

5. This final goal could be seen as an outcome of training. 

6. Some projects selected third and fourth ranked goals, but the lack of such 
goals across all projects make these responses difficult to analyze. 

7. This response probably means that the project provided computer-assisted basic 
skills instruction. 

8. We have not discussed these projects at length because additional inquiry 
about these projects yielded information that suggests high school students may 
not have been the primary clientele served. 

9. Of course, since many of these projects also report the fewest training hours 
per person, even if minorities were well represented they would not have received 
much training. 

10. Although described as basic skills in this item, the instruction is described 
as academic remediation elsewhere in the questionnaire. 

II. It is likely that this number does not include projects in which students 
were eligible for regular State or Federal financial aid, as an additional four 
projects indicated that financial aid was available through coordination. 
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